A Study of US Eighth Grade Mathematics

PURSUING EXCELLENCE

___

A STUDY OF U.S. EIGHTH-GRADE MATHEMATICS AND SCIENCE TEACHING, LEARNING, CURRICULUM, AND ACHIEVEMENT IN INTERNATIONAL CONTEXT

___

INITIAL FINDINGS FROM THE THIRD INTERNATIONAL MATHEMATICS AND SCIENCE STUDY

1 U.S. Department of Education Richard W. Riley Secretary

Office of Educational Research and Improvement Sharon P. Robinson Assistant Secretary

National Center for Education Statistics Pascal D. Forgione, Jr. Commissioner of Education Statistics

National Center for Education Statistics The National Center for Education Statistics (NCES) is the primary federal entity for collecting, analyzing, and reporting data related to education in the United States and other nations. It fulfills a congressional mandate to collect, collate, analyze, and report full and complete statistics on the condi- tion of education in the United States; conduct and publish reports and specialized analyses of the meaning and significance of such statistics; assist state and local education agencies in improving their statistical systems; and review and report on education activities in foreign countries.

NCES activities are designed to address high priority education data needs; provide consistent, reliable, complete, and accurate indicators of education status and trends; and report timely, useful, and high quality data to the U.S. Department of Education, the Congress, the states, other education policymakers, practitioners, data users, and the general public.

We strive to make our products available in a variety of formats and in language that is appropriate to a variety of audiences. You, as our customer, are the best judge of our success in communicating information effectively. If you have any comments or suggestions about this or any other NCES product or report, we would like to hear from you. Please direct your comments to: National Center for Education Statistics Office of Educational Research and Improvement U.S. Department of Education 555 New Jersey Avenue NW Washington, DC 20208-5574 Contact: (202) 219-1395

Suggested Citation: U.S. Department of Education. National Center for Education Statistics, Pursuing Excellence, NCES 97-198, Washington D.C.: U.S. Government Printing Office, 1996.

November 1996

Available for downloading at http://www.ed.gov/NCES/timss

2 C O M M I S S I O N E R ’ S S T A T E M E N T

With data on half a million students from 41 countries, the Third Interna- tional Mathematics and Science Study (TIMSS) is the largest, most comprehensive, and most rigorous international study of schools and students ever. This report, Pursuing Excellence, is a synthesis of initial findings from TIMSS on U.S. eighth-grade mathematics and science education, providing a comparative picture of education in the United States and the world that can be used to examine our education system, scrutinize improvement plans, and evaluate proposed standards and curricula. Subsequent TIMSS reports will examine U.S. mathematics and science education for fourth and twelfth-grade students in an international context.

TIMSS is significant not only because of its scope and magnitude, but also because of innovations in its design. In this international study, the National Center for Education Statistics (NCES) combined multiple methodologies to create an information base that goes beyond simple student test score comparisons and questionnaires to examine the fundamental elements of schooling. In- novative research techniques include analyses of textbooks and curricula, videotapes, and ethnographic case studies. The result is a more complete and accurate portrait of how U.S. mathematics and science education differs from that of other nations, with extended comparisons to Germany and Ja- pan.

The information in these reports can serve as a starting point for our efforts to define a “world-class” education. If the U.S. is to improve the mathematics and science education of its students, we must carefully examine not just how other countries rank, but also how their polices and practices help student achieve. TIMSS shows us where U.S. education stands — not just in terms of test scores, but also what is included in textbooks, taught in schools, and learned by students. Examining these data provides a unique opportunity to shed new light on education in the United States through the prism of other countries. As the same time, we should avoid the temptation to zero in on any one finding or leap to a conclusion without carefully considering the broader context.

This report is only the first of many NCES investigations into TIMSS data. Additional reports will be released throughout the coming year, including linkages of student achievement in U.S. states to achievement in the TIMSS countries, as well as findings on fourth and twelfth grade students. More- over, NCES plans to make TIMSS the most accessible international education study ever by releasing the data to scholars and the research community, and actively disseminating the findings to policymakers, educators, parents, and others concerned with quality education. Beginning with this report, NCES is releasing the information in a variety of new forms, including CD-ROM, videotape, and the World-Wide Web. Visit the NCES TIMSS website at “http://www.ed.gov/NCES/timss” for further information.

3 In all these efforts, our purpose is not just to take a snapshot of the present, but to develop a valuable resource for school improvement efforts. TIMSS clearly and accurately provides a wealth of useful data and information on curriculum, instruction, teacher and student lives, and student achievement. The investment in TIMSS can enhance the quality of our nation’s mathematics and science education, and improve the performance of our students to a more internationally competitive level.

Pascal D. Forgione Jr. Commissioner of Education Statistics

4 TABLE OF CONTENTS

EXECUTIVE SUMMARY ...... 9

PREFACE ...... 12 Key Points ...... 12 Overview ...... 13 Study design ...... 14 The TIMSS research team ...... 16 Organization of this report ...... 17

CHAPTER 1: ACHIEVEMENT ...... 18 Key Points ...... 18 How well do U.S. students do? ...... 19 Some special notes on the test scores ...... 19 Which countries outperform the U.S. in both subjects? ...... 23 Which countries does the U.S. outperform in both subjects? ...... 24 How do we compare to our major economic partners? ...... 24 How far behind the top countries are we? ...... 25 How do our best students compare with others’ best? ...... 25 How does the U.S. mathematics and science gender gap compare to other countries’? ...... 26 How do we score in the different content areas of mathematics and science? ...... 27 What did prior studies show about how U.S. states compare to other countries? ...... 32 Has U.S. international standing improved over time? ...... 32

CHAPTER 2: CURRICULUM ...... 34 Key Points ...... 34 Who sets curriculum standards? ...... 35 Is curriculum in the U.S. as focused as in other countries? ...... 36 Is curriculum in the U.S. as advanced as in other countries? ...... 38 How much time is spent in class? ...... 38

CHAPTER 3: TEACHING ...... 40 Key Points ...... 40 How do mathematics teachers structure and deliver their lessons?...... 42 Is the mathematical content of U.S. lessons as rich as that in Germany and Japan? ...... 44

5 To what extent are the recommendations of the mathematics reform movement being implemented? ...... 46 What do initial findings show about science teaching? ...... 47

CHAPTER 4: TEACHERS’ LIVES ...... 48 Key Points ...... 48 Who teaches mathematics and science? ...... 49 How do teachers spend their time? ...... 50 How do teachers learn to teach? ...... 51 What challenges do teachers face? ...... 53

CHAPTER 5: STUDENTS’ LIVES ...... 56 Key Points ...... 56 What does the system require of students? ...... 57 How do students spend their time during school? ...... 60 How much study do students do after school? ...... 62 What do students think about mathematics and science? ..... 65 What do students do after school besides study? ...... 66

CONCLUSIONS ...... 68 Key Points ...... 68 Where do we stand? ...... 69 What have we learned about mathematics? ...... 69 What have we learned about science? ...... 70 What have we learned about U.S. education as a whole? ...... 71 Questions for further study ...... 71 TIMSS’ long-term utility to the nation ...... 72

WORKS CITED ...... 73

APPENDIX 1: Additional TIMSS Reports ...... 74

APPENDIX 2: Advisors to the U.S. TIMSS Study ...... 76

APPENDIX 3: National Average Scores and Standard Errors ...... 78

APPENDIX 4: Summary of National Deviations from International Study Guidelines ...... 79

6 L I S T O F F I G U R E S

FIGURE 1: Nations’ Average Mathematics Performance Compared to the U.S...... 20

FIGURE 2: Nations’ Average Science Performance Compared to the U.S...... 21

FIGURE 3: Average Achievement of Nations Meeting, and Not Meeting International Guidelines ...... 22

FIGURE 4: Percent of Students from Selected Nations Scoring Among the Top 10 Percent of Eighth-Graders in the 41 TIMSS Countries ...... 26

FIGURE 5: National Averages in Mathematics Content Areas ...... 28

FIGURE 6: National Averages in Science Content Areas...... 30

FIGURE 7: Number of TIMSS Countries Determining Curriculum at Various Levels ...... 35

FIGURE 8: Hours of Mathematics and Science Instructional Time Per Year for Eighth Graders ...... 39

FIGURE 9: Comparison of the Steps Typical of Eighth-Grade Mathematics Lessons in Japan, the U.S., and Germany ...... 42

FIGURE 10: Average Percentage of Topics in Eighth-Grade Mathematics Lessons that Are Stated or Developed ...... 44

FIGURE 11: Expert Judgments of the Quality of the Mathematical Content of Eighth-Grade Lessons ...... 45

FIGURE 12: Percentage of Mathematics Teachers of TIMSS Students Reporting that Various Circumstances Limit Their Teaching “Quite a Lot” or “A Great Deal” ...... 52

FIGURE 13: Discipline Problems Eighth-Grade Principals Deal with on a Daily Basis ...... 54

FIGURE 14: Percent of Mathematics and Science Teachers Who Assign Homework 3-5 Times Per Week ...... 62

FIGURE 15: Percent of Eighth-Graders Spending 3 or More Hours in Various After-School Activities on a Normal School Day ...... 65

7 8 EXECUTIVE SUMMARY

PREFACE ■ In science, U.S. eighth graders The Third International Mathematics score above the international av- and Science Study (TIMSS) is the larg- erage of 41 TIMSS countries. Our est, most comprehensive, and most rig- students’ scores are not signifi- orous international comparison of edu- cantly different from those of cation ever undertaken. During the Canada, England, and Germany. 1995 school year, the study tested the math and science knowledge of a half- ■ In mathematics, our eighth-grade million students from 41 nations at five students’ standing is at about the different grade levels. In addition to international average in Algebra; tests and questionnaires, it included a Fractions; and Data Representa- curriculum analysis, videotaped obser- tion, Analysis, and Probability. We vations of mathematics classrooms, and do less well in Geometry; Measure- case studies of policy issues. ment; and Proportionality.

■ TIMSS’ rich information allows us ■ In science, our eighth graders’ not only to compare achievement, standing is above the international but also provides insights into how average in Earth Science, Life Sci- life in U.S. schools differs from that ence, and Environmental Issues. in other nations. Our students score about average in Chemistry and Physics. ■ This report on eighth-grade students

is one of a series of reports that will ■ If an international talent search were also present findings on student to select the top 10 percent of all achievement at fourth grade, and at students in the 41 TIMSS countries, the end of high school, as well as on in mathematics 5 percent of U.S. various other topics. students would be included. In science 13 percent would be included.

ACHIEVEMENT CURRICULUM One of our national goals is to be “first in the world in mathematics and sci- U.S. policy makers are concerned ence achievement by the year 2000,” about whether expectations for our as President Bush and 50 governors students are high enough, and in declared in 1989. Although we are particular whether they are as chal- far from this mark, we are on a par lenging as those of our foreign eco- with other major industrialized na- nomic partners. In all countries, the tions like Canada, England, and Ger- relationship between standards, many. teaching, and learning is complex. This is even more true in the U.S., ■ In mathematics, U.S. eighth grad- which is atypical among TIMSS coun- ers score below the international av- tries in its lack of a nationally defined erage of the 41 TIMSS countries. curriculum. Our students’ scores are not significantly different from those of En- gland and Germany.

9 ■ The content taught in U.S. eighth- ■ U.S. mathematics teachers’ typi- grade mathematics classrooms is cal goal is to teach students how at a seventh-grade level in com- to do something, while Japanese parison to other countries. teachers’ goal is to help them understand mathematical concepts. ■ Topic coverage in U.S. eighth- grade mathematics classes is not ■ Japanese teachers widely practice as focused as in Germany and what the U.S. mathematics reform Japan. recommends, while U.S. teachers do so infrequently. ■ In science, the degree of topic focus in the U.S. eighth-grade curriculum may ■ Although most U.S. math teachers be similar to that of other countries. report familiarity with reform recommendations, only a few apply the ■ U.S. eighth graders spend more key points in their classrooms. hours per year in math and science classes than German and Japanese students. TEACHERS’ LIVES

The training that teachers receive be- TEACHING fore they enter the profession and the regular opportunities that they have for In recent years, concern about the qual- on-the-job learning and improvement ity of instruction in U.S. classrooms of their teaching affect the quality of has led mathematics professional orga- the teaching force. The collegial sup- nizations to issue calls for reform. How- port that teachers receive and the char- ever, TIMSS data cannot tell us about acteristics of their daily lives also af- the success of these reform efforts for fect the type of teaching they provide. several reasons, including the fact that this assessment occurred too soon af- ■ Unlike new U.S. teachers, new Japa- ter the beginning of the reform for nese and German teachers undergo states and districts to have designed long-term structured apprentice- their own programs, retrained teach- ships in their profession. ers, and nurtured a generation of students according to the new approach. ■ U.S. teachers have more college edu- Also, we do not have comparable ear- cation than their colleagues in all but lier baseline information against which a few TIMSS countries. to compare the findings from TIMSS. However, TIMSS includes the first large- ■ Japanese teachers have more oppor- scale observational study of U.S. teach- tunities to discuss teaching-related ing ever undertaken, and this can form issues than do U.S. teachers. a baseline against which future progress may be judged. ■ Student diversity and poor discipline are challenges not only for U.S. ■ U.S. mathematics classes require stu- teachers, but for German teachers dents to engage in less high-level as well. mathematical thought than classes in Germany and Japan.

10 STUDENTS’ LIVES CONCLUSIONS

The manner in which societies structure This report presents initial findings from the schooling process gives rise to differ- TIMSS for eighth-grade mathematics ent opportunities and expectations for and science. A fuller understanding of young people. The motivators, supports, our nation’s educational health must and obstacles to study in each country await data from the fourth and twelfth- are outgrowths of the choices provided grade levels. The search for factors as- by society and schools. sociated with student performance is complicated because student achieve- ■ Eighth-grade students of different ment after eight years of schooling is the abilities are typically divided into dif- product of many different factors. Fur- ferent classrooms in the U.S., and thermore, the U.S. education system is into different schools in Germany. large and decentralized with many in- In Japan, no ability grouping is prac- terrelated parts. No single factor in iso- ticed at this grade level. lation from others should be regarded ■ In mathematics, U.S. students in as the answer to improving the perfor- higher ability-level classes study dif- mance of U.S. eighth-grade students. ferent material than students in With these cautions in mind, this report lower-level classes. In Germany and offers the following insights into factors Japan, all students study basically the that may be associated with our stu- same material, although in Germany dents’ performance: the depth and rigor of study depends ■ The content of U.S. eighth-grade on whether the school is for students mathematics classes is not as chal- of higher or lower ability levels. lenging as that of other countries,

■ Japanese eighth-graders are prepar- and topic coverage is not as focused. ing for a high-stakes examination to ■ Most U.S. mathematics teachers re- enter high school at the end of ninth port familiarity with reform recom- grade. mendations, only a few apply the key ■ U.S. teachers assign more homework points in their classrooms. and spend more class time discuss- ■ Evidence suggests that U.S. teaching it than teachers in Germany and ers do not receive as much practical Japan. U.S. students report about training and daily support as their the same amount of out-of-school German and Japanese colleagues. math and science study as their Japa- nese and German counterparts. TIMSS is not an answer book, but a

■ Heavy TV watching is as common mirror through which we can see our among U.S. eighth graders as it is own education system in international among their Japanese counterparts. perspective. Careful study of our nation’s reflection in the mirror of international comparisons will assist educators, business leaders, teachers, and parents as they guide our nation in the pursuit of excellence.

11 PREFACE

KEY POINTS:

The Third International Mathematics and Science Study (TIMSS) is the largest, most comprehensive, and most rigorous international comparison of education ever undertaken.

TIMSS’ rich information allows us not only to compare achievement, but also to understand how life in U.S. schools differs from that in other nations.

This report on eighth-grade students is the first of a series of reports that will present findings on student achievement at the fourth grade, at the end of high school, as well as on various other topics.

12 OVERVIEW for reform in the teaching of their subjects. The National Council of Teachers The Third International Mathematics of Mathematics published Curriculum and Science Study is the largest and and Evaluation Standards in 1989, and most comprehensive comparative in- Professional Standards for Teaching Math- ternational study of education that ematics in 1991. In 1993 the American has ever been undertaken. A half- Association for the Advancement of Sci- million students from 41 countries ence followed suit with Benchmarks for were tested in 30 different languages Science Literacy, and in 1996, the National at five different grade levels to com- Academy of Sciences published National pare their mathematics and science Science Education Standards. achievement. Intensive studies of students, teachers, schools, curricu- TIMSS helps us measure progress to- lum, instruction, and policy issues ward our national goal of improving our were also carried out to understand children’s academic performance in the educational context in which mathematics and science. But TIMSS learning takes place. is much more than a scorecard for the math and science events in the “educational Olympics.” It is a diagnostic TIMSS COUNTRIES tool to help us examine our nation’s

AUSTRALIA KOREA progress toward improvement of math- AUSTRIA KUWAIT ematics and science education. It was BELGIUM (FLEMISH) LATVIA BELGIUM (FRENCH) LITHUANIA designed to look behind the scorecard BULGARIA NETHERLANDS CANADA NEW ZEALAND to illuminate how our education poli- COLOMBIA NORWAY cies and practices compare to those of CYPRUS PORTUGAL CZECH REPUBLIC ROMANIA the world community. DENMARK RUSSIAN FEDERATION ENGLAND SCOTLAND FRANCE SINGAPORE TIMSS helps us answer the following GERMANY SLOVAK REPUBLIC GREECE SLOVENIA questions about our nation’s mathemat- HONG KONG SOUTH AFRICA HUNGARY SPAIN ics and science learning: ICELAND SWEDEN IRAN, ISLAMIC REPUBLIC SWITZERLAND IRELAND THAILAND ■ Are U.S. curricula and expectations ISRAEL UNITED STATES JAPAN for student learning as demanding as those of other nations?

TIMSS is an important study for those ■ Is the level of classroom instruction interested in U.S. education. In 1983, in the U.S. as high as that in other the National Commission on Excellence countries? in Education pointed to our nation’s low performance in international studies as ■ Do U.S. teachers receive as much evidence that we were A Nation at Risk. support in their efforts to teach stu- In 1989, President Bush and the gover- dents as their colleagues in other nors of all 50 states adopted the Na- nations? tional Goals for Education, one of which was that “by the year 2000, the U.S. will ■ Are U.S. students as focused on their be first in the world in mathematics and studies as their international coun- science achievement.” Mathematics and terparts? science experts have issued major calls

13 This report draws from the many reports the 41 countries to make sure that the and parts of the TIMSS study to summa- instructions were properly followed. The rize the initial findings concerning achieve- raw data from each country were scruti- ment and schooling in the eighth grade. nized to be sure that no anomalies ex- It is part of the first of three waves of isted, and all analyses were double TIMSS reports. It will be followed in the checked. Finally, this report has been next year by a series of reports focusing written and carefully reviewed to avoid on the fourth grade, then by a series fo- over-generalization and inaccuracy. cusing on the last year of high school. Additional reports on selected topics will be published over the next several years. Much more will be learned as further STUDY DESIGN analysis of the eighth grade data is car- TIMSS is the third comparison of ried out and findings from grades four and mathematics and science achievement twelve are added. carried out by the International Asso- TIMSS is a fair and accurate compari- ciation for the Evaluation of Educa- son of mathematics and science tional Achievement (IEA). Previous achievement in the participating na- IEA studies of mathematics and sci- tions. It is not a comparison of “all of ence were conducted for each subject our students, with other nations’ best,” separately at various times during the a charge which some critics have lev- 1960s, 1970s, and 1980s. This is the eled at previous international compari- first time that IEA has assessed both sons. The students who participated mathematics and science in the same in TIMSS were randomly selected to study. Comparative studies of other represent all students in their respec- subjects, including reading literacy 1 tive nations. The entire assessment (1992) , and computers in education 2 process was scrutinized by international (1993) have also been published by the technical review committees to ensure IEA. its adherence to established standards. Those nations in which irregularities TIMSS was designed to focus on stu- arose are clearly noted in this and other dents at three different stages of school- TIMSS reports. ing: midway through elementary school, midway through lower secondary school, At each step of its development, TIMSS and at the end of upper secondary used careful quality control procedures. school. Because countries around the An international curriculum analysis was world set different ages at which chil- carried out prior to the development of dren should begin school, decisions the assessments to ensure that the tests about which students should be tested reflect the math and science curricula needed to take both age and grade level of the variety of TIMSS countries and into account. The populations tested do not over-emphasize what is taught are listed below. Participation in Popu- in only a few. International monitors lation 2 was required of all TIMSS na- carefully checked the test translations tions, but participation in Populations and visited many classrooms while the 1 and 3 was optional. tests were being administered in each of

14 ■ Population 1 - those students enrolled ing results based on different research in the pair of adjacent grades that methods. Second, it provides broader contained the most nine-year-olds. information because more different (Grades 3 and 4 in the U.S. and most types data are gathered than can be ac- of the world. Grades 2 and 3 in a quired through a single method or in- few nations.) strument. Third, the use of multiple ■ Population 2 - those students in the methodologies enriches understanding pair of adjacent grades that con- of the contextual meaning of key find- tained the most thirteen-year-olds ings. Each of the five parts on its own at the time of testing. (Grades 7 and represents an important advance in its 8 in the U.S. and most of the world. field. Taken together, they provide an Grades 6 and 7 in a few nations.) unprecedented opportunity to under- ■ Population 3 - students in their fi- stand U.S. mathematics and science edu- nal year of secondary school, what- cation from a new and richer perspec- ever their age. (Grade 12 in the U.S. tive. and most nations. Grades 9-13 in some nations.) At population 2, all 41 TIMSS countries participated in the following three In all countries, students in both public IEA-sponsored parts of the study: and private schools received the TIMSS test. In all but a few of the 41 TIMSS ■ Math and science assessments - One countries, virtually all population 1 and and a half hours in length, the as- 2 children are enrolled in school and sessments included both multiple- were therefore eligible to take the test. choice and free-response items. A Testing occurred 2 to 3 months before smaller number of students also the end of the 1995-96 school year. Stu- completed “hands-on” performance dents with special needs and disabilities assessments, to be reported later. which would make it difficult for them to take the test were excused from the ■ School, teacher, and student ques- assessment. In each country, the test tionnaires - Students answered was translated into the primary language questions about their mathematics or languages of instruction. All testing and science studies and beliefs. in the U.S. was done in the English lan- Teachers answered questions on guage. their beliefs about math and science and on teaching practices. TIMSS includes five different parts: as- School administrators answered sessments, questionnaires, curriculum questions about school policies analyses, videotapes of classroom in- and practices. struction, and case studies of policy topics. The study was designed to bring a ■ Curriculum analysis - This ex- variety of different and complementary ploratory study compared math- research methods to bear on important ematics and science curriculum policy questions. The use of multiple guides and textbooks. It studied methodologies has three major benefits. subject-matter content, sequenc- First, it strengthens the conclusions of ing of topics, and expectations for the study because researchers are able student performance. to cross-check key findings by compar-

15 In conjunction with these three activi- can be found in the additional reports ties, the United States sponsored two listed in Appendix 1. additional parts of TIMSS, which were carried out in Germany, Japan, and the U.S. These three countries are all economic superpowers with close eco- THE TIMSS RESEARCH TEAM nomic and political ties. They also are TIMSS was conducted by the IEA, nations whose educators have learned which is a Netherlands-based organi- a great deal from each other in the past, zation of ministries of education and and whose school systems are both simi- research institutions in its member lar to and different from each other in countries. The IEA delegated respon- important ways. The TIMSS research- sibility for overall coordination and ers in Germany, Japan, and U.S. collabo- management of the TIMSS study to rated in sharing their assessment and Professor Albert Beaton at the TIMSS questionnaire data, and in carrying out International Study Center, located at the following two parts of the study: Boston College. Each of the 41 IEA member-nations that made the deci- ■ Videotapes of mathematics instruction - In the U.S. and Germany, half sion to participate in TIMSS paid for of the eighth-grade mathematics and carried out the data collection in classrooms that participated in the its own country according to the in- main TIMSS study were randomly ternational guidelines. The costs of the chosen to be filmed. In Japan, an international coordination were paid eighth-grade classroom in a random by the National Center for Education sample of 50 of the TIMSS schools Statistics of the U.S. Department of was chosen to be videotaped. In all Education (NCES), the National Sci- three countries teachers were filmed ence Foundation (NSF), and the Cana- teaching a typical lesson, and these dian Government. tapes were analyzed to compare TIMSS in the United States was also teaching techniques and the quality funded by NCES and NSF. Professor of instruction. William Schmidt of Michigan State Uni- versity was the U.S. National Research ■ Ethnographic case studies of key Coordinator. Policy decisions on the policy topics - A team of 12 bilin- study were made by the U.S. National gual researchers each spent three Coordinating Committee, composed of months in Germany, Japan, or the William Schmidt, Larry Suter of NSF, U.S. observing classrooms and inter- and Jeanne Griffith, Eugene Owen, and viewing education authorities, prin- Lois Peak of NCES. Lois Peak moni- cipals, teachers, students, and par- tored the international and U.S. TIMSS ents. Topics of study were education data collections. The U.S. data collec- standards, methods of dealing with tion was carried out by Westat, a pri- individual differences, the lives and vate survey research firm. Trevor Will- working conditions of teachers, and iams and Nancy Caldwell were Westat the role of school in adolescents’ lives. project co-directors. Professor James More detail on the findings and meth- Stigler at UCLA managed the TIMSS odology of each of these parts of TIMSS videotape study of mathematics instruc-

16 tion, and Professor Harold Stevenson at In some respects, results for mathemat- the University of Michigan managed the ics and science are similar. The report TIMSS ethnographic case studies. The focuses more on mathematics for two many advisors to the study are listed in reasons. First, the way in which the sub- Appendix 2. ject is taught makes it easier to compare across countries. Second, TIMSS con- The U.S. TIMSS team also includes the tains more data about mathematics be- nearly 4,000 seventh and 7,000 eighth cause the videotapes of classroom in- graders who took the assessment, and struction were conducted only in this their principals and teachers in more subject. Discussion of findings notes than 180 schools nationwide. Their co- where the results in science differ operation has made this report possible. from those in math. This report de- Third, fourth, and twelfth graders also scribes the U.S. against the backdrop took different TIMSS tests, and find- of the 41 TIMSS countries, with a spe- ings from these parts of the study will cial attention to comparisons with be reported during the next year. Germany and Japan, because we have more information on these countries. ORGANIZATION OF THIS REPORT Chapter 1 draws from the results of This report summarizes early findings the student assessments to describe how from the eighth-grade data based on U.S. students perform in mathematics results from all five parts of the TIMSS and science. Succeeding chapters fo- study. Both seventh and eighth grade cus on factors which may have an im- students took the TIMSS test, but this portant influence on achievement, and initial report focuses on findings for describe how our nation’s schools, the eighth grade. Future reports based teachers, and students compare to those on a more complete and extensive in other countries. Chapter 2 exam- analysis of the data will provide deeper ines educational standards and the cur- understanding and investigate relation- riculum, based on data from the cur- ships between the findings from the riculum analysis, case studies, video- different parts of the study. Science tape study, and questionnaires. teacher questionnaire data used in this Chapter 3 focuses on how teachers report are based on preliminary weights actually teach that curriculum, drawing which will be further refined in subse- from results of the videotape study and quent reports. Extensive documentation questionnaires. Chapter 4 examines the of the data collection methodologies and working life of teachers, based upon statistical analyses used in this report are findings from the case studies and ques- available from NCES, and will be pub- tionnaires. Chapter 5 describes the lives lished separately. of students, both in and out of school, based upon case study and questionnaire This report combines the major find- data. The Conclusions at the end of the ings from each of the five parts of the report looks across all of the findings for study into a single story about U.S. insights about factors associated with eighth-grade mathematics and science student performance and indicates ques- achievement in comparative perspective. tions for further study.

17 CHAPTER 1: ACHIEVEMENT

KEY POINTS:

U.S. eighth graders score below average in mathematics achievement and above average in science achievement, compared to the 41 nations in the TIMSS assessment.

In mathematics, our eighth-grade students’ international standing is stronger in Algebra and Fractions than in Geometry and Measurement.

In science, our eighth graders’ international standing is stronger in Earth Science, Life Science, and Environmental Issues than in Chemistry and Physics.

The U.S. is one of 11 TIMSS nations in which there is no significant gender gap in eighth- grade math and science achievement.

18 In the past, the mathematics and science dence that comparisons of other coun- achievement of U.S. students has caused tries to the U.S. are accurate plus or nation-wide cries for improvement. Vari- minus about 20 points, depending on ous international studies of these sub- the size and design of the sample in the jects conducted over the past thirty years other countries. Comparisons of the have shown that our eighth graders have U.S. to the international average are not performed as well as we expect, in accurate plus or minus about 10 points. comparison to their peers in other na- (Appendix 3 contains a list of standard tions. U.S. students are not weak in all errors). Because the precise score can- subjects, however. In a recent IEA study not be determined with perfect accuracy, of reading literacy3, U.S. eighth graders to fairly compare the U.S. to other coun- were among the best in the world. In- tries, nations have been grouped into deed, TIMSS shows that U.S. eighth broad bands according to whether grade students also scored better than their performance is higher than, not the average of the 41 participating coun- significantly different from, or lower tries in science. The results in math- than the U.S. ematics, however, put our nation below average compared to the other nations. In mathematics, students in 20 countries outperform our eighth graders. Students in 13 countries are not sig- HOW WELL DO U.S. STUDENTS nificantly different than ours, and U.S. DO? students outperform their counterparts in 7 nations. In science, students in 9 Compared to their international coun- nations outperform U.S. eighth grad- terparts, U.S. students are somewhat ers, performance in 16 other nations is below the international average of 41 not statistically different than ours, and TIMSS countries in mathematics. In we outperform another 15 nations. science, our students are somewhat above the international average. Fig- ures 1 and 2 on pages 20 and 21 show SOME SPECIAL NOTES ON THE how U.S. students perform in these sub- TEST SCORES jects. TIMSS required participating nations to Tempting as it may be, it is not correct to adhere to extremely high technical stan- report U.S. scores by rank alone, as dards at all stages of participation in would be the case if one were to say the project. Many nations experienced the U.S. is “number x in mathemat- some difficulty in this respect. In two ics out of the 41 TIMSS countries.” nations, difficulties in meeting the stan- This is because the process of esti- dards were so severe that international mating each country’s score from the monitors decided that their data should sample of students who took the test not be included in the report, and so produces only an estimate of the findings are reported only for the remain- range within which the country’s real ing 41 nations. Of the 41 nations, 25 score lies. This margin of error is ex- met or came close to meeting all tech- pressed as a “plus or minus” interval nical standards for the study. However, around the estimated score. In TIMSS, 16 nations experienced difficulties of we can say with 95 percent confi- various types. In some countries, these

19 FIGURE 1: NATIONS’ AVERAGE MATHEMATICS PERFORMANCE COMPARED TO THE U.S.

NATIONS WITH AVERAGE SCORES NATIONS WITH AVERAGE SCORES SIGNIFICANTLY HIGHER THAN THE U.S. NOT SIGNIFICANTLY DIFFERENT FROM THE U.S. NATION AVERAGE NATION AVERAGE SINGAPORE 643 (THAILAND) 522 KOREA 607 (ISRAEL) * 522 JAPAN 605 (GERMANY) * O 509 HONG KONG 588 NEW ZEALAND 508 BELGIUM-FLEMISH O 565 ENGLAND * O 506 CZECH REPUBLIC 564 NORWAY 503 SLOVAK REPUBLIC 547 (DENMARK) 502 SWITZERLAND O 545 UNITED STATES O 500 (NETHERLANDS) 541 (SCOTLAND) 498 (SLOVENIA) 541 LATVIA (LSS) O 493 (BULGARIA) 540 SPAIN 487 (AUSTRIA) 539 ICELAND 487 FRANCE 538 (GREECE) 484 HUNGARY 537 (ROMANIA) 482 RUSSIAN FEDERA TION 535 (AUSTRALIA) 530 IRELAND 527 NATIONS WITH AVERAGE SCORES CANADA 527 SIGNIFICANTLY LOWER THAN THE U.S. (BELGIUM-FRENCH) 526 NATION AVERAGE SWEDEN 519 LITHUANIA * 477 CYPRUS 474 INTERNATIONAL AVERAGE= 513 PORTUGAL 454 IRAN, ISLAMIC REPUBLIC 428 SOURCE: Beaton et al. (1996) Mathematics achievement (KUWAIT) 392 in the middle school years. Table 1.1. Boston College: Chest- (COLOMBIA) 385 nut Hill, MA. (SOUTH AFRICA) 354

NOTES: 1. Nations not meeting international guidelines are shown in parentheses. 2. Nations in which more than 10 percent of the population was excluded from testing are shown with a *. Latvia is designated LSS because only Latvian-speaking schools were tested, which represents less than 65 percent of the population. 3. Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after replacements for refusals were substituted, are shown with a o. 4. The international average is the average of the national averages of the 41 nations. 5. The country average for Sweden may appear to be out of place; however, statistically, its placement is correct. difficulties arose because a large propor- slightly older than the international tar- tion of schools, teachers, or students de- get age. The names of those nations in clined to take the test. In others, the which major difficulties arose are shown selection of schools or classrooms was in parentheses in the figures in this re- not carried out according to interna- port, and Appendix 4 describes any de- tional plan. In still others, students were viations from international specifica- (continued on page 23)

20 FIGURE 2: NATIONS’ AVERAGE SCIENCE PERFORMANCE COMPARED TO THE U.S.

NATIONS WITH AVERAGE SCORES NATIONS WITH AVERAGE SCORES SIGNIFICANTLY HIGHER THAN THE U.S. SIGNIFICANTLY LOWER THAN THE U.S.

NATION AVERAGE NATION AVERAGE SINGAPORE 607 SPAIN 517 CZECH REPUBLIC 574 FRANCE 498 JAPAN 571 (GREECE) 497 KOREA 565 ICELAND 494 (BULGARIA) 565 (ROMANIA) 486 (NETHERLANDS) 560 LATVIA (LSS) O 485 (SLOVENIA) 560 PORTUGAL 480 (AUSTRIA) 558 (DENMARK) 478 HUNGARY 554 LITHUANIA * 476 (BELGIUM-FRENCH) 471 IRAN, ISLAMIC REPUBLIC 470 NATIONS WITH AVERAGE SCORES CYPRUS 463 NOT SIGNIFICANTLY DIFFERENT FROM THE U.S. (KUWAIT) 430 (COLOMBIA) 411 NATION AVERAGE (SOUTH AFRICA) 326 ENGLAND * O 552 BELGIUM-FLEMISH O 550 INTERNATIONAL AVERAGE= 516 (AUSTRALIA) 545 SOURCE: Beaton et al. (1996) Science achievement in SLOVAK REPUBLIC 544 the middle school years. Table 1.1. Boston College: Chest- RUSSIAN FEDERA TION 538 nut Hill, MA. IRELAND 538 NOTES: SWEDEN 535 1. Nations not meeting international guidelines are shown in parentheses. O UNITED STATES 534 2. Nations in which more than 10 percent of the popu- (GERMANY) * O 531 lation was excluded from testing are shown with a *. CANADA 531 Latvia is designated LSS because only Latvian-speaking schools were tested, which represents less than 65 NORWAY 527 percent of the population. NEW ZEALAND 525 3. Nations in which a participation rate of 75 percent of the schools and students combined was achieved only (THAILAND) 525 after replacements for refusals were substituted, are (ISRAEL) * 524 shown with a o. HONG KONG 522 4. The international average is the average of the national averages of the 41 nations. O SWITZERLAND 522 5. The country average for Scotland (or Spain) may ap- (SCOTLAND) 517 pear to be out of place; however, statistically, its placement is correct.

21 FIGURE 3: AVERAGE ACHIEVEMENT OF NATIONS MEETING, AND NOT MEETING, INTERNATIONAL GUIDELINES

COUNTRIES COMPLYING COUNTRIES WITH LOW WITH SPECIFICATIONS PARTICIPATION RATES

NATION MATH SCIENCE NATION MATH SCIENCE AVERAGE AVERAGE AVERAGE AVERAGE BELGIUM-FLEMISH O 565 550 AUSTRALIA 530 545 CANADA 527 531 AUSTRIA 539 558 CYPRUS 474 463 BELGIUM-FRENCH 526 471 CZECH REPUBLIC 564 574 BULGARIA 540 565 ENGLAND * O 506 552 NETHERLANDS 541 560 FRANCE 538 498 SCOTLAND 498 517 HONG KONG 588 522 HUNGARY 537 554 COUNTRIES TESTING ICELAND 487 494 OLDER-THAN-SPECIFIED STUDENTS IRAN, ISLAMIC REPUBLIC 428 470 IRELAND 527 538 NATION MATH SCIENCE JAPAN 605 571 AVERAGE AVERAGE KOREA 607 565 COLOMBIA 385 411 LATVIA (LSS) O 493 485 GERMANY 509 531 LITHUANIA * 477 476 ROMANIA 482 486 NEW ZEALAND 508 525 SLOVENIA 541 560 NORWAY 503 527 PORTUGAL 454 480 COUNTRIES WITH NON-STANDARD RUSSIAN FEDERA TION 535 538 SELECTION OF CLASSROOMS SINGAPORE 643 607 SLOVAK REPUBLIC 547 544 NATION MATH SCIENCE SPAIN 487 517 AVERAGE AVERAGE SWEDEN 519 535 DENMARK 502 478 SWITZERLAND O 545 522 GREECE 484 497 UNITED STATES O 500 534 THAILAND 522 525 INTERNATIONAL AVERAGE = 527 527 COUNTRIES WITH NON-STANDARD SELECTION OF CLASSROOMS AND Notes: OTHER DEPARTURES FROM GUIDELINES 1. Nations in which more than 10 percent of the population was excluded from testing are shown with a *. Latvia is designated LSS because only Latvian-speaking schools NATION MATH SCIENCE were tested, which represents less than 65 percent of AVERAGE AVERAGE the population. ISRAEL 522 524 2. Nations in which a participation rate of 75 percent of the schools and students combined was achieved only KUWAIT 392 430 after replacements for refusals were substituted, are SOUTH AFRICA 354 326 shown with a o. 3. The international average is 527 for both mathematics and science. This is the average of the national averages of Source: Beaton et al. (1996) Mathematics achievement the 25 countries meeting international guidelines. in the middle school years. Table 1.1. Boston College: Chest- 4. The international average based on all 41 countries listed nut Hill, MA., and Beaton et al. (1996) Science achievement is 513 for mathematics and 516 for science. in the middle school years. Table 1.1. Boston College: Chest- nut Hill, MA.

22 tions that occurred. It should be kept What do the test scores mean? Due to in mind that we cannot have the same the complex nature of the TIMSS test amount of confidence in the scores of design, scoring, and analysis, a score of the 16 nations in which major difficul- 600 does not mean either 600 items, or ties arose. 60 percent correct. One can interpret the scores by considering where they fall If the international average is calcu- along the range of scores from 0 to 1000 lated only from the 25 countries in of other eighth-grade students who took which no major difficulties arose in the test. In mathematics, a score of 656 carrying out the international speci- would put a student in the top 10 per- fications, the U.S. mathematics cent of all students in the 41 TIMSS score is still below the international countries, and a score of 587 would put average. In science, however, our a student in the top 25 percent. In math- score is no longer significantly dif- ematics, 509 was the average student ferent from the average of the 25 na- score. In science, a score of 655 would tions. Our comparative position in put a student in the top 10 percent, a science becomes lower because many score of 592 would put a student in the of the countries who are removed from top 25 percent, and 522 was the aver- consideration are those that we out- age student score. performed. Figure 3 on page 22 shows our mathematics and science standing in comparison to these 25 nations, and the types of anomalies that occurred in WHICH COUNTRIES the other 16 countries. The difference OUTPERFORM THE U.S. IN BOTH in U.S. standing between Figure 3 and SUBJECTS? the previous figures demonstrates that the selection of countries against which We can say with confidence that five the U.S. is compared can change our in- nations outperformed us in both math- ternational standing. ematics and science. They are:

Which comparison should we empha- ■ Three Asian nations - Singapore, size as TIMSS’ main finding – the com- Korea, and Japan. parison to 25 countries, or to 41? ■ Two Central European nations - NCES has chosen as the primary find- Czech Republic and Hungary. ing our standing with respect to 41 The Netherlands, Austria, Slovenia, and countries because the international Bulgaria also outperformed us in both TIMSS reports present the results in subjects, but because these countries terms of all 41 nations. did not carry out TIMSS according to strict international standards, we can be less certain about their scores. These nine countries were the only ones that outperformed us in science, and they were also among the 20 countries that outperformed us in mathematics.

23 WHICH COUNTRIES DOES THE U .S. those of England and Germany. In sci- OUTPERFORM IN BO TH SUBJECTS? ence, we score lower than Japan; were not significantly different than England, We can say with confidence that the Canada, and Germany; and score higher U.S. outperformed four countries in than France. Considering our stand- both mathematics and science: ing in relation to these five major economic partners, it can be said that the ■ Three European countries - U.S. is in the bottom half in math- Lithuania, Cyprus, and Portugal. ematics, and about the middle in sci- ■ One Middle Eastern country - Iran. ence.

The U.S. also outperformed Kuwait, Co- Among the G-7 countries, Germany is lombia, and South Africa in both sub- the only nation which appears in pa- jects, but due to deviations in their ad- rentheses, indicating problems in the ministration of TIMSS, we have less implementation of the international confidence in their scores. These seven guidelines for carrying out the study. countries were the only ones that we In Germany, the problem was a discrep- outperformed in mathematics, and ancy in the age of the students tested. they were also among the 15 countries Because German children start school that we outperformed in science. somewhat later than children in other countries, the average age of the Ger- HOW DO WE COMP ARE TO OUR man eighth-graders who took the MAJOR ECONOMIC P ARTNERS? TIMSS test was about four months older than the international target age. The “Group of Seven” or G-7 countries Some would say that this means that are major U.S. economic and political other nations’ eighth graders should be allies. The other six nations in this group compared with Germany’s seventh grad- are the United Kingdom, France, Ger- ers for a better age comparison. How- many, Canada, Japan, and Italy. Italy ever, this provides a less-than-ideal grade did not administer the TIMSS test, so comparison. the U.S. can only be compared to the remaining five. The United Kingdom In a grade-based comparison, there includes England, Scotland, Northern is no significant difference between Ireland, and Wales. Northern Ireland German and U.S. eighth graders. If and Wales did not participate in TIMSS, we were to approximate an age-based and England and Scotland both have the comparison by matching the scores same international standing in compari- of our eighth graders to those of Ger- son to the U.S. Therefore, in this sec- man seventh graders, our eighth tion, we describe our standing in rela- graders do significantly better. Both tion to England. comparisons are useful because most experts believe that achievement is In mathematics, Japan, France, and based partly on cognitive maturation Canada outperform the U.S., while our which comes with age, and partly on scores are not significantly different from years of study which come with grade in school.

24 HOW FAR BEHIND THE T OP ence between U.S. and Japanese eighth COUNTRIES ARE WE? graders’ mathematics performance is about four times this difference. Particularly in mathematics, our students are far behind Singapore and In science, the gap is smaller, but still Japan which are among the top-scor- substantial. The difference between U.S. ing nations in the world in both math seventh and U.S. eighth graders’ scores and science. One way to compare two is 26 points. The difference between the nations’ scores is by considering their scores of the U.S. and Singapore was 73 comparative standing with relation to points. The difference in science per- the international percentiles. In math- formance between eighth graders in the ematics, the scores of our very best U.S. U.S. and Singapore is almost three times eighth graders, who perform at the 95th the difference between our seventh and percentile for our nation, are not signifi- eighth graders. The difference between cantly different than the scores of aver- U.S. and Japanese eighth graders’ sci- age eighth graders in Singapore, who per- ence performance is almost one and a form at their nation’s 50th percentile. In half times this difference. comparison to Japan, the scores of our best students, who are at the 95th percentile for our nation, are significantly below the scores of the top quarter of HOW DO OUR BEST STUDENTS Japanese students, who perform at their COMPARE WITH OTHERS’ BEST? nation’s 75th percentile. Comparisons of averages tell us how typical students perform, but they do In science, the gap is not so large. Stu- not tell us about the performance of dents at the U.S. 95th percentile are our nation’s best students - those who significantly better than students at the are likely to become the next genera- 75th percentile in Singapore. In com- tion of mathematicians, scientists, parison to Japan, there is no significant doctors, and engineers. If an interna- difference between U.S. and Japanese students at the 95th percentile. tional talent search were to select the top ten percent of all students in Another way to estimate distance be- the 41 TIMSS countries combined, tween the U.S. and top scoring coun- what percentage of U.S. students tries is to use the difference between our would be included? seventh and our eighth graders as a unit of measure. In mathematics, the differ- In mathematics, 5 percent of U.S. ence between our seventh and eighth eighth graders would be selected. High-scoring nations would have more graders’ scores was 24 points. The differ- of their students represented in the “in- ence between the scores of eighth grad- ternational top ten percent.” Figure 4 ers in the U.S. and in Singapore was 143 on page 26 shows that 45 percent of all points. This means that the difference Singaporean students and 32 percent of in eighth-grade mathematics perfor- all Japanese students would be chosen mance between the two countries is al- in the international talent search in most six times the difference between U.S. mathematics. In science, 13 percent seventh and eighth graders. The differ-

25 FIGURE 4: of U.S. students would be selected, in PERCENT OF STUDENTS FROM SELECTED comparison to 31 percent of NATIONS SCORING AMONG THE TOP 10 Singaporean students and 18 percent of PERCENT OF EIGHTH GRADERS IN THE 41 Japanese students. TIMSS COUNTRIES If the international talent search were 45 SINGAPORE to lower its standards considerably to 31 choose the top half of all students in the 41 TIMSS countries, 94 percent of eighth graders in Singapore and 83 per- 34 KOREA cent in Japan would be selected in 18 mathematics, compared to 45 percent of eighth graders in the U.S. In science, 82 percent of the students in 32 JAPAN Singapore and 71 percent of students 18 in Japan would be selected, compared to 55 percent in the U.S.

CZECH 18 REPUBLIC 19 HOW DOES THE U.S. MATHEMATICS AND SCIENCE 11 GENDER GAP COMPARE TO HUNGARY 14 OTHER COUNTRIES’?

In the U.S. and in other countries, RUSSIAN 10 policy makers have made great efforts FEDERATION 11 to make math and science more accessible to girls, and to encourage gender equity in these subjects. More TIMSS 7 ENGLAND countries have achieved gender equity 17 in their students’ scores in mathematics than in science. The U.S. is one of 7 11 TIMSS nations in which there is FRANCE no significant gender gap in eighth- 1 grade mathematics and science MATHEMATICS achievement. The U.S. was one of 33 7 SCIENCE CANADA countries in which there was no statisti- 9 cally significant difference between the performance of eighth-grade boys and 6 GERMANY 11 Source: Beaton et al. (1996) Mathematics achievement in the middle school years. Table 1.4. Boston College: Chest- UNITED 5 nut Hill, MA., and Beaton et al. (1996) Science achievement STATES in the middle school years. Table 1.4. Boston College: Chest- 13 nut Hill, MA. 0 5 10 15 20 25 30 35 40 45 50

26 girls in mathematics. In science, we were ■ Geometry (visualization and proper- one of 11 nations with no statistically ties of geometric figures, including significant difference. All 11 nations symmetry, congruence, and similar- with no significant different in science ity). also demonstrated no difference in mathematics. They are the United States, ■ Measurement (units of length, weight, Singapore, the Russian Federation, Thai- time, area, volume, and interpretation land, Australia, Ireland, Romania, Flem- of measurement scales). ish Belgium, Cyprus, Columbia, and South Africa. ■ Proportionality (proportionality and ratios).

Figure 5 on pages 28 and 29 shows that HOW DO WE SCORE IN THE among these content areas, U.S. stu- DIFFERENT CONTENT AREAS OF dents’ performance is at about the in- MATHEMATICS AND SCIENCE? ternational average in Algebra; Data Representation, Analysis, and Prob- Representing student achievement in ability; and Fractions and Number mathematics and science as a total Sense. Compared to other countries, score is a useful way to summarize we do less well in Geometry; Measure- achievement. However, mathematics ment; and Proportionality. Our weaker and science contain different content performance in these latter three topics areas, which are emphasized and se- may pull the overall U.S. score down to quenced differently in curricula around below average. the world. Based on these national priorities, in each country, some content areas have been studied more than oth- In science, the TIMSS eighth-grade test ers at a particular grade level. sampled students’ ability to do work in the following subjects: The TIMSS eighth-grade mathematics test included sets of items designed to ■ Chemistry (classification of matter, sample students’ ability to do work in chemical properties and transforma- the following areas: tions).

■ Algebra (patterns, relations, expres- ■ Earth Science (earth features, earth sions, equations). processes, and the earth in the uni- verse). ■ Data Representation, Analysis, and Prob- ability (representation and analysis ■ Environmental Issues and the Nature of of data using charts and graphs in- Science (environmental and resource volving uncertainty and probability). issues, the nature of scientific knowledge, and the interaction of science ■ Fractions and Number Sense (fractions, and technology). decimals, percentages, estimation and rounding). (Continued on page 32) .

27 FIGURE 5: NATIONAL AVERAGES IN MATHEMATICS CONTENT AREAS

FRACTIONS & NUMBER GEOMETRY ALGEBRA SENSE

NATION PERCENT CORRECT NATION PERCENT CORRECT NATION PERCENT CORRECT SINGAPORE 84 JAPAN 80 SINGAPORE 76 JAPAN 75 SINGAPORE 76 JAPAN 72 KOREA 74 KOREA 75 HONG KONG 70 HONG KONG 72 HONG KONG 73 KOREA 69 BELGIUM-FLEMISH O 71 CZECH REPUBLIC 66 CZECH REPUBLIC 65 CZECH REPUBLIC 69 FRANCE 66 HUNGARY 63 SWITZERLAND O 67 (BULGARIA) 65 RUSSIAN FEDERATION 63 SLOVAK REPUBLIC 66 BELGIUM-FLEMISH O 64 BELGIUM-FLEMISH O 63 (AUSTRIA) 66 RUSSIAN FEDERATION 63 SLOVAK REPUBLIC 62 IRELAND 65 SLOVAK REPUBLIC 63 (BULGARIA) 62 HUNGARY 65 (THAILAND) 62 (SLOVENIA) 61 FRANCE 64 (SLOVENIA) 60 (ISRAEL) * 61 CANADA 64 HUNGARY 60 (AUSTRIA) 59 (SLOVENIA) 63 SWITZERLAND O 60 (AUSTRALIA) 55 SWEDEN 62 (NETHERLANDS) 59 SPAIN 54 (BELGIUM-FRENCH) 62 (BELGIUM-FRENCH) 58 FRANCE 54 RUSSIAN FEDERATION 62 CANADA 58 CANADA 54 (NETHERLANDS) 62 (AUSTRALIA) 57 IRELAND 53 (AUSTRALIA) 61 (ISRAEL) * 57 (BELGIUM-FRENCH) 53 (ISRAEL) * 60 (AUSTRIA) 57 (THAILAND) 53 (BULGARIA) 60 LATVIA (LSS) O 57 SWITZERLAND O 53 56 (THAILAND) 60 NEW ZEALAND 54 (NETHERLANDS) 53 UNITED STATES O 59 ENGLAND *O 54 (ROMANIA) 52 52 (GERMANY) *O 58 (DENMARK) 54 UNITED STATES O 51 58 NORWAY 58 LITHUANIA * 53 LATVIA (LSS) O 51 NEW ZEALAND 57 (ROMANIA) 52 NEW ZEALAND 49 ICELAND 54 (SCOTLAND) 52 ENGLAND *O 49 ENGLAND *O 54 IRELAND 51 (GERMANY) *O 48 (SCOTLAND) 53 (GERMANY) *O 51 CYPRUS 48 (DENMARK) 53 ICELAND 51 LITHUANIA * 47 (GREECE) 53 NORWAY 51 (SCOTLAND) 46 LATVIA (LSS) O 53 (GREECE) 51 (GREECE) 46 SPAIN 52 SPAIN 49 NORWAY 45 LITHUANIA * 51 SWEDEN 48 (DENMARK) 45 CYPRUS 50 UNITED STATES O 48 SWEDEN 44 (ROMANIA) 48 CYPRUS 47 ICELAND 40 PORTUGAL 44 PORTUGAL 44 PORTUGAL 40 IRAN, ISLAMIC REPUBLIC39 IRAN, ISLAMIC REPUBLIC43 IRAN, ISLAMIC REPUBLIC37 (COLOMBIA) 31 (KUWAIT) 38 (KUWAIT) 30 (KUWAIT) 27 (COLOMBIA) 29 (COLOMBIA) 28 (SOUTH AFRICA) 26 (SOUTH AFRICA) 24 (SOUTH AFRICA) 23

NOTES: 1. Nations not meeting international study guidelines are shown in parentheses. 2. Nations in which more than 10 percent of the population was excluded from testing are shown with a *. Latvia is designated LSS because only Latvian-speaking schools were tested, which represents less than 65 percent of the population. 3. Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after replacements for refusals were substituted, are shown with a o. 4. The international average is the average of the national averages of the 41 nations.

28 DATA REPRESENTATION, ANALYSIS, MEASUREMENT PROPORTIONALITY & PROBABILITY

NATION PERCENT CORRECT NATION PERCENT CORRECT NATION PERCENT CORRECT SINGAPORE 79 SINGAPORE 77 SINGAPORE 75 KOREA 78 JAPAN 67 HONG KONG 62 JAPAN 78 KOREA 66 KOREA 62 BELGIUM-FLEMISH O 73 HONG KONG 65 JAPAN 61 SWITZERLAND O 72 CZECH REPUBLIC 62 BELGIUM-FLEMISH O 53 (NETHERLANDS) 72 (AUSTRIA) 62 SWITZERLAND O 52 HONG KONG 72 SWITZERLAND O 61 CZECH REPUBLIC 52 FRANCE 71 SLOVAK REPUBLIC 60 (NETHERLANDS) 51 SWEDEN 70 BELGIUM-FLEMISH O 60 (THAILAND) 51 IRELAND 69 (SLOVENIA) 59 IRELAND 51 CANADA 69 (NETHERLANDS) 57 (SLOVENIA) 49 (AUSTRIA) 68 FRANCE 57 SLOVAK REPUBLIC 49 CZECH REPUBLIC 68 HUNGARY 56 (AUSTRIA) 49 (BELGIUM-FRENCH) 68 RUSSIAN FEDERATION 56 FRANCE 49 (AUSTRALIA) 67 SWEDEN 56 RUSSIAN FEDERATION 48 (DENMARK) 67 (BELGIUM-FRENCH) 56 CANADA 48 NORWAY 66 (BULGARIA) 54 (BELGIUM-FRENCH) 48 NEW ZEALAND 66 (AUSTRALIA) 54 HUNGARY 47 (SLOVENIA) 66 IRELAND 53 (BULGARIA) 47 ENGLAND *O 66 NORWAY 51 (AUSTRALIA) 47 45 HUNGARY 66 CANADA 51 51 SWEDEN 44 UNITED STATES O 65 (GERMANY) *O 51 (ISRAEL) * 43 (SCOTLAND) 65 (THAILAND) 50 NEW ZEALAND 42 (GERMANY) *O 64 ENGLAND *O 50 UNITED STATES O 42 (ISRAEL) * 63 (DENMARK) 49 (GERMANY) *O 42 ICELAND 63 NEW ZEALAND 48 (ROMANIA) 42 (THAILAND) 63 (ISRAEL) * 48 ENGLAND *O 41 (BULGARIA) 62 (SCOTLAND) 48 (DENMARK) 41 62 SLOVAK REPUBLIC 62 (ROMANIA) 48 NORWAY 40 RUSSIAN FEDERATION 60 LATVIA (LSS) O 47 SPAIN 40 SPAIN 60 ICELAND 45 (SCOTLAND) 40 (GREECE) 56 SPAIN 44 CYPRUS 40 LATVIA (LSS) O 56 CYPRUS 44 (GREECE) 39 PORTUGAL 54 (GREECE) 43 LATVIA (LSS) O 39 CYPRUS 53 LITHUANIA * 43 ICELAND 38 LITHUANIA * 52 UNITED STATES O 40 IRAN, ISLAMIC REPUBLIC36 (ROMANIA) 49 PORTUGAL 39 LITHUANIA *O 35 IRAN, ISLAMIC REPUBLIC41 IRAN, ISLAMIC REPUBLIC29 PORTUGAL 32 (KUWAIT) 38 (COLOMBIA) 25 (COLOMBIA) 23 (COLOMBIA) 37 (KUWAIT) 23 (SOUTH AFRICA) 21 (SOUTH AFRICA) 26 (SOUTH AFRICA) 18 (KUWAIT) 21

SOURCE: PERCENT SIGNIFICANTLY HIGHER THAN U.S. Beaton et al. (1996) Mathematics PERCENT NOT SIGNIFICANTLY DIFFERENT THAN U.S. achievement in the middle school years. Table 2.1. Boston College: Chestnut PERCENT SIGNIFICANTLY LOWER THAN U.S. Hill, MA. # INTERNATIONAL AVERAGE PERCENT CORRECT, ALL NATIONS

29 FIGURE 6: NATIONAL AVERAGES IN SCIENCE CONTENT AREAS

EARTH LIFE PHYSICS SCIENCE SCIENCE

NATION PERCENT CORRECT NATION PERCENT CORRECT NATION PERCENT CORRECT SINGAPORE 65 SINGAPORE 72 SINGAPORE 69 (SLOVENIA) 64 JAPAN 71 JAPAN 67 CZECH REPUBLIC 63 KOREA 70 KOREA 65 KOREA 63 CZECH REPUBLIC 69 CZECH REPUBLIC 64 O BELGIUM-FLEMISH 62 (NETHERLANDS) 67 (NETHERLANDS) 63 (AUSTRIA) 62 (THAILAND) 66 (AUSTRIA) 62 O SWEDEN 62 HUNGARY 65 ENGLAND * 62 NORWAY 61 (AUSTRIA) 65 SLOVAK REPUBLIC 61 IRELAND 61 (SLOVENIA) 65 (SLOVENIA) 61 O (NETHERLANDS) 61 (BULGARIA) 64 BELGIUM-FLEMISH 61 O JAPAN 61 ENGLAND * 64 (BULGARIA) 60 O SLOVAK REPUBLIC 60 BELGIUM-FLEMISH 64 (AUSTRALIA) 60 HUNGARY 60 (AUSTRALIA) 63 HUNGARY 60 O O ENGLAND * 59 (GERMANY) * 63 CANADA 59 O RUSSIAN FEDERATION 58 UNITED STATES 63 HONG KONG 58 (BULGARIA) 58 SWEDEN 63 NEW ZEALAND 58 O O UNITED STATES 58 RUSSIAN FEDERATION 62 SWITZERLAND 58 O SWITZERLAND 58 CANADA 62 RUSSIAN FEDERATION 57 O CANADA 58 HONG KONG 61 (GERMANY) * 57 (AUSTRALIA) 57 NORWAY 61 SWEDEN 57 O (GERMANY) * 57 (ISRAEL) * 61 (ISRAEL) * 57 SPAIN 57 NEW ZEALAND 60 (SCOTLAND) 57 (THAILAND) 56 SLOVAK REPUBLIC 60 NORWAY 57 NEW ZEALAND 56 IRELAND 60 IRELAND 56 O O (ISRAEL) * 55 SWITZERLAND 59 59 UNITED STATES 56 55 FRANCE 55 ICELAND 58 SPAIN 55 55 HONG KONG 54 SPAIN 58 FRANCE 54 (SCOTLAND) 52 (SCOTLAND) 57 (THAILAND) 54 PORTUGAL 50 FRANCE 56 ICELAND 53 (BELGIUM-FRENCH) 50 (DENMARK) 56 (GREECE) 53 ICELAND 50 (ROMANIA) 55 (DENMARK) 53 (ROMANIA) 49 (BELGIUM-FRENCH) 55 (BELGIUM-FRENCH) 51 O (GREECE) 49 (GREECE) 54 LATVIA (LSS) 51 (DENMARK) 49 PORTUGAL 53 LITHUANIA * 51 O O LATVIA (LSS) 48 LATVIA (LSS) 53 (ROMANIA) 49 LITHUANIA * 46 LITHUANIA * 52 PORTUGAL 48 CYPRUS 46 IRAN, ISLAMIC REPUBLIC 49 IRAN, ISLAMIC REPUBLIC 48 IRAN, ISLAMIC REPUBLIC 45 CYPRUS 49 CYPRUS 46 (KUWAIT) 43 (KUWAIT) 45 (KUWAIT) 43 (COLOMBIA) 37 (COLOMBIA) 44 (COLOMBIA) 37 (SOUTH AFRICA) 26 (SOUTH AFRICA) 27 (SOUTH AFRICA) 27

30 NOTES: ENVIRONMENTAL ISSUES 1. Nations not meeting international CHEMISTRY & THE NATURE study guidelines are shown in pa- OF SCIENCE rentheses. 2. Nations in which more than 10 NATION PERCENT CORRECT NATION PERCENT CORRECT percent of the population was ex- SINGAPORE 69 SINGAPORE 74 cluded from testing are shown (BULGARIA) 65 (NETHERLANDS) 65 with a *. Latvia is designated LSS O KOREA 63 ENGLAND * 65 because only Latvian-speaking schools were tested, which repre- JAPAN 61 KOREA 64 sents less than 65 percent of the CZECH REPUBLIC 60 (AUSTRALIA) 62 population. HUNGARY 60 (THAILAND) 62 O 3. Nations in which a participation (AUSTRIA) 58 UNITED STATES 61 rate of 75 percent of the schools SLOVAK REPUBLIC 57 CANADA 61 and students combined was RUSSIAN FEDERATION 57 IRELAND 60 achieved only after replacements (SLOVENIA) 56 JAPAN 60 for refusals were substituted, are o SWEDEN 56 (BULGARIA) 59 shown with a . O ENGLAND * 55 CZECH REPUBLIC 59 4. The international average is the average of the national averages of HONG KONG 55 NEW ZEALAND 59 O the 41 nations. (GERMANY) * 54 (SLOVENIA) 59 O IRELAND 54 BELGIUM-FLEMISH 58 SOURCE: (AUSTRALIA) 54 (SCOTLAND) 57 Beaton et al. (1996) Science achieve- (ISRAEL) * 53 NORWAY 55 ment in the middle school years. Table O UNITED STATES 53 HONG KONG 55 2.1. Boston College: Chestnut Hill, NEW ZEALAND 53 (AUSTRIA) 55 MA. (NETHERLANDS) 52 SLOVAK REPUBLIC 53 IRAN, ISLAMIC REPUBLIC 52 HUNGARY 53 53 CANADA 52 FRANCE 53 SPAIN 51 SPAIN 53 PERCENT SIGNIFICANTLY (GREECE) 51 (ISRAEL) * 52 O 51 BELGIUM-FLEMISH 51 SWEDEN 52 HIGHER THAN U.S. O (SCOTLAND) 51 (GERMANY) * 51 PERCENT NOT SIGNIFICANTLY O PORTUGAL 50 SWITZERLAND 51 DIFFERENT FROM U.S. O SWITZERLAND 50 (GREECE) 51 PERCENT SIGNIFICANTLY LOWER NORWAY 49 RUSSIAN FEDERATION 50 O THAN U.S.INTERNATIONAL LATVIA (LSS) 48 ICELAND 49 LITHUANIA * 48 (DENMARK) 47 # AVERAGE PERCENT CORRECT O FRANCE 47 LATVIA (LSS) 47 (ROMANIA) 46 CYPRUS 46 CYPRUS 45 (BELGIUM-FRENCH) 46 (THAILAND) 43 PORTUGAL 45 ICELAND 42 (ROMANIA) 42 (BELGIUM-FRENCH) 41 (COLOMBIA) 40 (DENMARK) 41 LITHUANIA * 40 (KUWAIT) 40 (KUWAIT) 39 (COLOMBIA) 32 IRAN, ISLAMIC REPUBLIC 39 (SOUTH AFRICA) 26 (SOUTH AFRICA) 26

31 ■ Life Science (structure, diversity, clas- which most other TIMSS national gov- sification, processes, cycles, and in- ernments are responsible for their own teractions of plants and animals). education system.

■ Physics (energy forms, physical trans- Future analyses may make possible such formations, force and motion, and comparisons between U.S. states and physical properties of matter). the TIMSS nations. Efforts are now underway to create an experimental linkage between the TIMSS study and Figure 6 on pages 30 and 31 shows our the mathematics and science portions of comparative standing in these content the National Assessment of Educational areas. The U.S. is among the top coun- Progress (NAEP). This linkage will al- tries in the world in Environmental Is- low an estimation of how states would sues and the Nature of Science, and we have performed on TIMSS if their stu- are also above the international average dents had taken the test. The results in Earth Science and Life Science. In for eighth-grade mathematics and sci- Chemistry and Physics, our perfor- ence will be announced in 1997. mance is not significantly different from the international average. Our Until those findings are released, how- better-than-average scores in Environ- ever, we can look at the results of a mental Issues, Earth Science, and Life similar linkage which was performed Science may pull our overall science in 1991 for eighth-grade mathematics score up to above average. students’ scores on NAEP and on the International Assessment of Educa- tional Progress4. In that comparison, the mathematics scores of Iowa, WHAT DID PRIOR STUDIES SHOW North Dakota, and Minnesota were ABOUT HOW U .S. STATES similar to top-scoring Taiwan and COMPARE TO OTHER COUNTRIES? Korea. In contrast, Alabama, Loui- siana, and Mississippi scored about Comparison of U.S. states with other the same as lowest-scoring Jordan. nations reminds us that not all U.S. These findings underscore the con- school systems are alike, and that wide siderable variation in achievement differences in achievement exist that exists among states within our own within our own nation. Some would nation. say that comparisons of U.S. states and other nations are fair for two reasons. First, most U.S. states are larger either HAS U.S. INTERNATIONAL in size or population than many coun- STANDING IMPROVED OVER tries in the TIMSS study. For example, TIME? California is larger in size than Japan, Germany, or England. New Jersey has a Results from the National Assess- larger population than Austria, Den- ment of Education Progress show that mark, or Switzerland. A second reason our eighth-grade students’ scores in that such comparisons are fair is that math and science have improved some- each U.S. state is responsible for its own education system, similar to the way in

32 what in comparison to our own perfor- In TIMSS mathematics, we have seen mance during the past decades. If our that our eighth-graders scored below the domestic performance over time is im- international average. This is basically proving, how does this affect our inter- the same relative international standing national standing? It is possible that reported for U.S. thirteen-year-olds in only U.S. achievement has improved the IEA First and Second International over time, while achievement in other Mathematics Studies in the 1960s and countries has not. Of course, it is also 1980s, and the mathematics portion of possible that improvements in the U.S. the International Assessment of Educa- have been matched or exceeded by im- tional Progress in the early 1990s5. Rela- provements in other countries. tive to their international counterparts, it is not likely that U.S. eighth- International comparisons over time are graders’ standing in mathematics has difficult. The first international studies improved significantly. of math and science achievement were conducted in the 1960s, and there have In the three previous international been three previous assessments in each science assessments in the 1960s, subject since that time. However, each 1980s, and early 1990s, the U.S. per- assessment has been done differently. A formed below the international average different set of nations participated, dif- of thirteen or fourteen-year-olds. How- ferent topics in math and science were ever in TIMSS, our students scored at included in the tests, the age and type or above the international average. Be- of students sampled in each country cause comparisons over time are dif- changed slightly, and indeed even the ficult, caution should be exercised in borders and names of some of the na- assuming there has been significant tions have changed. Furthermore, the improvement in our international field of assessment has matured greatly standing in science, but it is a possi- over the past thirty years, rendering the bility. methods of the then-revolutionary early studies crude by today’s standards. We have now examined what TIMSS These and other factors complicate com- tells us about what eighth-grade stu- parisons over time, and require that any dents have learned. Learning, of course, conclusions that are drawn be necessar- is closely related to what students are ily tentative. taught. Next we turn to an examination of how the U.S. mathematics and science curricula compare with those of other nations.

33 CHAPTER 2 : CURRICULUM

KEY POINTS:

The content taught in U.S. eighth-grade mathematics classrooms is at a seventh- grade level in comparison to other countries.

Topic coverage in U.S. eighth-grade mathematics classes is not as focused as in Germany and Japan.

In science, the degree of topic focus in the eighth-grade curriculum may be similar to that of other countries.

Our nation is atypical among TIMSS countries in its lack of a nationally-defined curriculum.

U.S. eighth graders spend more hours per year in math and science classes than German and Japanese students.

34 U.S. policy makers are concerned about teachers actually teach their students is whether standards for our students are sometimes called the “implemented cur- high enough, and, in particular, whether riculum.” Both the officially intended they are as challenging as those of our curriculum and the implemented cur- foreign economic partners. There is a riculum must be considered when dis- widespread belief that our nation’s eco- cussing a nation’s goals for student learn- nomic productivity is related to our stu- ing. dents’ performance in mathematics and science, and that this in turn is related to the expectations that are set for stu- WHO SETS CURRICULUM dent performance. STANDARDS?

However, the relationship between stan- In most TIMSS countries, the curricu- dards, teaching, and learning is not a lum is determined by national au- simple one. Formal and informal deci- thorities. Figure 7 shows that curricu- sions at many levels affect what stu- lum is determined at the national level dents are taught. National, state, and in 29 of the TIMSS countries, at the local authorities as well as publishers state or region in 3 countries, and at the set forth the officially intended curricu- local or district level in 9 countries. lum in both curriculum guidelines and Germany, Japan, and the U.S. differ in textbooks. Teachers also make deci- this respect, which makes comparisons sions about what should be taught. De- among the three countries interesting. pending on the country, their decisions Which authority sets a country’s offi- are based more or less closely on the of- cial curriculum standards makes a dif- ficially intended curriculum. What ference in whether or not there is a single

FIGURE 7 NUMBER OF TIMSS COUNTRIES DETERMINING CURRICULUM AT VARIOUS LEVELS

LOCAL LEVEL- 9 COUNTRIES INCLINCLUDING THE U.S. NATIONAL LEVEL- STATE LEVEL- 29 COUNTRIES 3 COUNTRIES INCLINCLUDING INCLINCLUDING JAPAN GERMANY

SOURCE: Beaton et al. (1996) Mathematics achievement in the middle school years. Figure 1. Boston College: Chestnut Hill, MA.

35 official core curriculum for the entire na- specificity, and the degree to which tion, or whether there are as many offi- schools and teachers are held account- cial curricula as there are states or dis- able for following them. Teachers in tricts in the country. states where curriculum guidelines are not highly specific, and where schools Japan is one of the countries that de- and districts are allowed to develop their termines curriculum at the national own secondary school exit examinations, level. The National Ministry of Educa- have considerable flexibility in determin- tion specifies one set of curriculum ing what and how they teach. guidelines that details the topics of study and the number of instructional hours In the U.S., most of the nearly16,000 required in every accredited elementary districts design their own curriculum and junior high school. For these school- or standards, usually within broad ing levels, it also approves textbooks guidelines issued by each of the 50 published by six commercial publishers. states. There are many different com- Textbooks resemble each other in con- mercially published textbooks. Because tent because they must be based closely most textbooks are designed with an on the national guidelines. Local school eye to sales in as many districts as pos- boards make only minor modifications sible, they include the content speci- to the national guidelines, and choose fied by the guidelines from a number of textbooks from among the approved list. different states. As a result, textbooks However, the Ministry itself does not usually contain much more material monitor whether or not the standards than a teacher can cover fully in a year. are adhered to, leaving the issue of Each of the many different textbooks oversight to the local boards of edu- includes somewhat different topics from cation. Teachers of each subject in a which teachers in various districts can school work together closely to be choose. Few states or districts closely sure that they cover the material in monitor or enforce compliance with the textbooks at approximately the state or district standards, and U.S. same depth and rate. This is partly teachers usually have the latitude to de- due to the oversight of local authorities, sign the content and pace of their and partly due to teachers’ desire that courses to suit their perception of their their students score well on the high- students’ needs. school entrance examination, which is based directly on the national curriculum. IS CURRICULUM IN THE U.S. In Germany, each of the 16 states AS FOCUSED AS IN OTHER sets its own curriculum standards COUNTRIES? for students. To encourage some degree of similarity across states, the na- Evidence from a variety of sources tional Conference of Ministers of Edu- in TIMSS shows us that the U.S. cation discusses various issues related mathematics curriculum is less fo- to standards and adopts broad recom- cused than that of other countries. mended guidelines concerning curricu- The U.S. science curriculum more lum, hours of instruction, and exami- closely resembles international prac- nation guidelines. State curriculum tices. standards vary widely in their level of

36 The TIMSS curriculum analysis stud- found that the set of 5 U.S. eighth-grade ied the officially intended curriculum by texts included more different topics asking U.S. curriculum experts to judge across all the texts than the set of texts which topics were recommended to be in Japan and Germany. taught at each grade level. Their judgments were compared with those of ex- Of course, not all teachers cover every perts in the other TIMSS countries. topic recommended by curriculum ex- This effort revealed that the number of perts, or included in textbooks. There- topics recommended to be covered in the fore, TIMSS also studied the imple- U.S. was greater than the international mented curriculum—what teachers ac- average at each of grades 1 through 8 tually cover in their classrooms. Using for mathematics. the same definitions of mathematics topics that the curriculum analysis used, the Textbooks are another aspect of the of- videotape study of eighth-grade math- ficially intended curriculum. Video- ematics lessons in Germany, Japan, and tapes of mathematics classes in Ger- the U.S. revealed that U.S. lessons in- many, Japan, and the U.S. showed that clude a greater number of topics. On textbooks were used during class in al- average, U.S. teachers taught 1.9 topics most half of U.S. lessons and a third of per lesson, compared with 1.6 in Ger- German lessons, but in only 2 percent many and 1.3 in Japan. The variety of of Japanese lessons. Teacher-developed topics was much wider in the U.S., too. worksheets were common in U.S. and Japanese lessons. In Japan, students also In science, the officially intended cur- use supplementary practice books which riculum as reflected by U.S. curricu- are usually purchased from the school lum experts’ recommendations about for use in home study. topics to be taught was close to the international average for grades 3 The TIMSS curriculum analysis com- through 8. Science experts in each pared the most commonly used text- country chose the three most common books in the various countries. For the textbooks used in their classrooms, U.S. portion of this analysis, mathemat- which were found to be used by 16 ics experts were asked to recommend the percent of students in the U.S., and 84 most commonly-used U.S. eighth-grade percent of students in Japan. textbooks in these subjects. The TIMSS questionnaire surveys of teachers found Thus, the evidence from a variety of that these chosen texts were indeed TIMSS sources reinforces the finding among the most widely used books in that our eighth-grade mathematics cur- the U.S, although they accounted for the riculum is less focused than the cur- textbooks used by only 28 percent of the ricula of other nations, if focus is de- students. This finding that the five rec- fined as number and variety of topics ommended textbooks covered a fairly in the intended and implemented cur- small proportion of students is an indi- riculum. Although less information is cation of the great diversity of textbooks available for science, U.S. curricular fo- in our country. In Japan, close to 90 cus may be more similar to the average percent of the students used one of the of the TIMSS countries in this subject. five most common textbooks. Analysis

37 IS CURRICULUM IN THE U .S. TIMSS does not have data to judge AS ADVANCED AS IN O THER whether the U.S. curriculum in science COUNTRIES? is as advanced as that of other countries because the videotape study was con- The U.S. mathematics curriculum is ducted only in mathematics. not as advanced as in Germany and Japan. Concerning the intended curriculum, analysis of textbooks found that Geometry occupied more space in the HOW MUCH TIME IS SPENT IN CLASS? German and Japanese books than in the U.S. texts. The Japanese textbooks also Lengthening the school year or school devoted more space to algebra than did day has often been proposed as a mea- the books studied by the majority of sure to improve U.S. students’ achieve- U.S. eighth graders, who are in non-al- ment, as it has been thought that U.S. gebra tracks. students spend less time at school than their international counterparts. The implemented curriculum in the U.S. TIMSS compared the amount of time is also less advanced than that of Germany that teachers report U.S. students and Japan. In the videotapes studied, 40 spend in mathematics and science percent of U.S. eighth-grade mathemat- classes with the amount of time re- ics lessons included arithmetic topics such ported for students in Germany and as whole number operations, fractions, Japan. In contrast to previous analy- and decimals, whereas these topics were ses, TIMSS carefully took into ac- much less common in Germany and Ja- count differences between countries pan. In contrast, German and Japanese in the length of the school year, school eighth-grade lessons were more likely to week, and class period, as well as differ- cover algebra and geometry. ences between the amount of time required for students in high and low The topics being taught in U.S. math- tracks. On this basis, the average ematics classrooms were at a seventh- number of hours per year that a stu- grade level in comparison to other coun- dent in each country spends in math- tries, while the topics observed in the ematics and science class was calculated. German and Japanese classrooms were at a high eighth-grade or even U.S. eighth-graders spend consider- ninth-grade level. This was discovered ably more hours per year in math- based on a comparison of the TIMSS ematics classes than their Japanese curriculum analysis and videotape stud- and German counterparts. U.S. stu- ies. The curriculum analysis asked ex- dents also spend much more time in sci- perts in each of the TIMSS countries ence classes than students in Japan. Fig- to report the grade level at which their ure 8 on page 39 shows the amount of country focused on various topics. time that students in the different coun- These findings were compared to the tries spend in math and science classes topics which the TIMSS videotape per year. U.S. students’ instructional study observed eighth-grade teachers in time is both longer and more com- Japan, Germany, and the U.S. to be ac- pressed, because it takes place within a tually teaching. school year of approximately 180 days, as compared to 188 in Germany and 220

38 in Japan. Of course, time spent in home- to the question of why U.S. students work, after-school classes, and out-of- are below the international average school study is also an important factor in mathematics. Instead, findings in learning, and findings concerning suggest that our students receive a these topics will be examined in Chap- less-advanced curriculum, which is ter 5. also less focused. Next we will consider how this curriculum is taught by Taken together, TIMSS curriculum-re- examining the findings concerning class- lated findings show that lack of suffi- room teaching. cient class time is not the easy answer

FIGURE 8 HOURS OF MATHEMATICS AND SCIENCE INSTRUCTIONAL TIME PER YEAR FOR EIGHTH-GRADERS

160 143 140 140 136 U.S. 117 GERMANY 120 114 JAPAN 100 90

AVERAGE HOURS PER YEAR 20

0 MATHEMATICS SCIENCE

SOURCE: Third International Mathematics and Science Study; unpublished tabulations, U.S., German, and Japanese school and teacher surveys; Westat, Inc., 1996.

39 CHAPTER 3 : TEACHING

KEY POINTS:

The content of U.S. mathematics classes requires less high-level thought than classes in Germany and Japan.

U.S. mathematics teachers’ typical goal is to teach students how to do something, while Japanese teachers’ goal is to help them understand mathematical concepts.

Japanese teachers widely practice what the U.S. mathematics reform recommends, while U.S. teachers do so less frequently.

Although most U.S. math teachers report familiarity with reform recommendations, only a few apply the key points in their classrooms.

40 During the past several years, mathemat- TIMSS nations, rather than those spe- ics professional organizations, concerned cifically recommended by the U.S. re- about the quality of instruction in U.S. form community. Second, TIMSS classrooms, have issued calls for reform. tested U.S. students in the spring of In 1989, the National Council of Teach- 1995, which was too soon after the pub- ers of Mathematics (NCTM) set forth lication of the reform documents for Curriculum and Evaluation Standards, fol- states and districts to have designed their lowed in 1991 by Professional Standards own reform programs, retrained teach- for Teaching Mathematics, and in 1995 by ers in the new practices, and nurtured a Assessment Standards. The essence of the generation of students according to the recommendations in these reform docu- new approach. Third, a proper evalua- ments is that instruction should be more tion requires matching “before and af- than mere mastery of facts and routine ter” measurements between which skills. It should require students to un- progress can be judged, and we have no derstand and apply mathematical con- prior measurement which matches cepts in new situations. TIMSS. For these reasons, TIMSS is not suitable as an evaluation. It should Publication and discussion of docu- be studied as a baseline measurement ments such as these, however, do not against which future progress can be change the behavior of all of America’s gauged. hundreds of thousands of mathematics teachers within a few years. Rec- Until TIMSS, no large nationally-rep- ommendations for major changes in resentative study had observed U.S. other areas of American life, such as classrooms to watch how teachers ac- improving health through regular exer- tually teach. To overcome this lack, cise and proper diet, have required de- and to understand how U.S. classroom cades of sustained effort by public health teaching compares to that of other organizations at all levels to assist in- countries, NCES added an innovative dividual citizens in changing in- new research methodology to the grained personal habits and atti- TIMSS project— videotaping and tudes. Indeed, the campaign still quantitative coding of a national continues. Changing our nation’s sample of eighth-grade mathematics habits of teaching and public atti- classes in Germany, Japan, and the tudes toward mathematics and sci- U.S. ence may also require a similarly long and concerted effort by many commit- In the U.S. and Germany, half of the ted people. eighth-grade mathematics classrooms in which students were sched- TIMSS was not designed as an evalua- uled to take the TIMSS test were ran- tion of the U.S. mathematics reform domly chosen to be filmed. In Japan, efforts described in the documents 50 classrooms from the schools in which listed above. There are three reasons the TIMSS test was administered were why TIMSS is unsuitable as such an chosen by the principal and officials at evaluation. First, because it is an inter- the National Institute for Educational national study, it was designed to mea- Research. Teachers whose classrooms sure those aspects of mathematics and were chosen and who agreed to partici- science knowledge and practice consid- pate were videotaped teaching a typical ered important by the majority of lesson. In this way, videotapes of 230

41 lessons were collected in the three coun- teachers about their goals for the lesson. tries combined. The videotapes were In contrast to expert recommendation then coded and analyzed to compare the that well-taught lessons focus on hav- teaching techniques and lesson content ing students think about and come to typical of the three countries. Teachers understand mathematical concepts, also completed a questionnaire concern- U.S. and German eighth-grade math- ing the lesson that was videotaped. The ematics teachers usually explained findings can be considered representa- that the goal of their lesson was to tive of the type of instruction received have students acquire particular by German, Japanese, and U.S. eighth- skills, i.e. to learn how to do something. grade mathematics students. The results Learning a skill, such as being able to provide a window on actual teaching in solve a certain type of problem, or using U.S. classrooms, and also show how U.S. a standard formula, was listed as the goal mathematics classes compare to those by about 60 percent of the U.S. and Ger- in Germany and Japan. man teachers, compared with 27 percent of the Japanese teachers. Japanese teachers’ goals were more likely to re- HOW DO MATHEMATICS TEACHERS semble the recommendations of U.S. STRUCTURE AND DELIVER THEIR reform experts. Mathematical thinking, LESSONS? such as exploring, developing, and understanding concepts, or discovering When studying what teachers do in multiple solutions to the same problems, their classrooms, we should first un- was described as the goal of the lesson derstand what they mean to do. by 71 percent of the Japanese teachers, Therefore, the videotape study asked

FIGURE 9: COMPARISON OF THE STEPS TYPICAL OF EIGHTH-GRADE MATHEMATICS LESSONS IN JAPAN, THE U.S., AND GERMANY

The emphasis on understanding is evident in the steps typical of Japa- nese eighth-grade mathematics lessons: ■ Teacher poses a complex thought-provoking problem. ■ Students struggle with the problem. ■ Various students present ideas or solutions to the class. ■ Class discusses the various solution methods. ■ The teacher summarizes the class’ conclusions. ■ Students practice similar problems.

In contrast, the emphasis on skill acquisition is evident in the steps common to most U.S. and German math lessons: ■ Teacher instructs students in a concept or skill. ■ Teacher solves example problems with class. ■ Students practice on their own while the teacher assists individual students.

SOURCE: Third International Mathematics and Science Study; unpublished tabulations, Videotape Classroom Study, UCLA, 1996.

42 compared with 29 percent of German veloped if it was proven, derived, or ex- and 24 percent of U.S. teachers. This plained in some detail. difference in goals is played out in the typical sequences of activities, or cultural Figure 10 on page 44 shows that U.S. scripts, which characterize mathematics teachers rarely developed concepts, in lessons in the three countries. Figure 9 contrast to German and Japanese on page 42 describes the steps typical teachers, who usually did. In Germany, of these cultural scripts. the teacher usually did the mental work in developing the concept, while The U.S. and German emphasis on the students listened or answered skills rather than understanding is short questions designed to add to the also carried over into the type of flow of the teacher’s explanation. Japa- mathematical work that students are nese teachers, however, designed the les- assigned to do at their desks during son in such a way that the students class. Students were coded as practic- themselves derived the concept from ing routine procedures if their seatwork their own struggle with the problem. required them to carry out a previously- learned solution method or procedure These findings from the videotape study on a routine problem. In the U.S., are corroborated by the TIMSS ques- 96 percent of seatwork time was tionnaire findings. Teachers were asked spent on routine procedures, in com- to choose activities that were charac- parison to 89 percent in Germany, teristic of their teaching from among and 41 percent in Japan. Students those listed on the questionnaire. U.S. were assigned to invent new solu- math teachers were more likely to re- tions, proofs, or procedures on their port asking students to practice com- own which require them to think and putational skills, in most or every class reason in 44 percent of Japanese, 4 than were their German and Japa- percent of German lessons, and less nese colleagues. Similarly, Japanese than 1 percent of U.S. lessons. teachers were more likely to report Clearly, Japanese students much they ask students to analyze relation- more often engage in the type of ships, write equations, explain their mathematical thinking recommended reasoning, and solve problems with by experts and the U.S. reform move- no obvious solution in most or every ment. class than teachers in the U.S. and Germany. When a lesson included a mathematical concept, it was usually simply Linking concepts used in one part of the stated in U.S. classrooms, whereas lesson to ideas or activities in another it was developed in Japanese and part of the lesson is believed by experts German ones. For example, consider to improve students’ ability to learn and a lesson on the Pythagorean theo- understand a subject in an integrated rem. When the concept is merely way. The videotape study found that stated, the teacher or a student might 96 percent of Japanese lessons included simply say “we find the length of the such explicit linkages in comparison to hypotenuse of a right triangle by us- about 40 percent of U.S. and German ing a2+ b2= c2.” In contrast, a con- lessons. Talking about such relationships cept was coded as having been de- may help make lessons more coherent

43 FIGURE 10 AVERAGE PERCENTAGE OF TOPICS IN EIGHTH-GRADE MATHEMATICS LESSONS THAT ARE STATED OR DEVELOPED

100 83 80 77 78 STATED DEVELOPED 60

40 23

20 22

AVERAGE PERCENT TOPICS 17

0 GERMANY JAPAN U.S. SOURCE: Third International Mathematics and Science Study; unpublished tabulations, Videotape Classroom Study, UCLA, 1996.

for students by showing them the rela- Taken together, these findings suggest tionships between ideas and activities that Japanese rather than U.S. or Ger- used in different parts of the lesson. man lessons more often resembled the recommendations of experts and the Interruptions present a threat to the co- U.S. reform movement. U.S. lessons herence of lesson activities. The study typically focused on acquiring math- found that the flow of mathematics les- ematical skills rather than conceptual sons was more frequently interrupted understanding, and were less coher- than in Germany and Japan. One U.S. ently presented. math lesson in four was temporarily halted by an outside interruption, typically a loudspeaker announcement, or a IS THE MATHEMATICAL CONTENT OF visitor at the door. In contrast, inter- U.S. LESSONS AS RICH AS THAT IN ruptions in German lessons were much GERMANY AND JAPAN? less common, and the Japanese lessons observed in the study never experienced As noted earlier, the U.S. eighth-grade outside interruptions. Interruptions mathematics curriculum focuses more coming from within the classroom were on arithmetic, while the German and also more common in U.S. mathemat- Japanese curricula focus more on geom- ics lessons, such as substantial discus- etry and algebra. Furthermore, U.S. sion of non-mathematical subjects like eighth graders are studying topics usu- recent sports events, or extended disci- ally learned at the seventh grade in most plinary incidents. In the U.S., 23 per- other TIMSS countries. cent of lessons were broken up in this way, compared to 9 percent in Japan, and How does the quality of the math- 4 percent in Germany. ematical reasoning used in U.S. classrooms compare with that in

44 Germany and Japan? Videotape re- reviewed to disguise any words such as searchers requested the assistance of 3 “yen,” or “football,” or other hints mathematics professors and one profes- which might indicate the country in sor of mathematics education in evalu- which the lesson was taught. Each ex- ating the quality of the mathematics con- pert first independently rated the over- tained in the videotaped lessons. This all quality of the mathematical content group of four experts was asked to judge of each lesson as either low, medium, or the quality of the “story” formed by the high. After comparing their ratings, they sequence of mathematical ideas in a ran- found high agreement among their judg- dom sample of 90 of the lessons divided ments. Figure 11 below shows their evenly among each of the three coun- judgments. tries. They studied such factors as the coherence of the sequencing, the type None of the U.S. lessons was consid- of reasoning required of students, the in- ered to contain a high-quality se- crease in cognitive complexity between quence of mathematical ideas, com- the beginning and end of the lesson, and pared to 30 percent of the Japanese, the way in which the problems and ex- and 23 percent of the German lessons. amples contributed to the lesson’s cen- Instead, the lowest rating was assigned tral concept. to the mathematical reasoning used in 87 percent of the U.S. lessons, in com- To ensure that the experts were not parison to 40 percent of the German unconsciously biased toward any and 13 percent of Japanese lessons. country, they were not allowed to actu- This finding does not mean that there ally see the videotapes. Instead, they are no lessons with high-quality were provided with a written summary mathematical reasoning anywhere in of each lesson’s sequence of mathemati- the U.S. However, it does indicate cal statements and equations, as well as that they are probably a rare phe- how these were embedded in learning nomenon. activities. The summaries were carefully

FIGURE 11 EXPERT JUDGEMENTS OF THE QUALITY OF THE MATHEMATICAL CONTENT OF EIGHTH-GRADE LESSONS 100 LOW 87 80 MEDIUM HIGH

60 57

40 40 37 30 23

PERCENT OF LESSONS 20 13 13 0 0 GERMANY JAPAN U.S. SOURCE: Third International Mathematics and Science Study; unpublished tabulations, Videotape Classroom Study, UCLA, 1996.

45 These findings that our nation’s eighth- and Professional Teaching Standards. grade mathematics classes are based on less challenging material, and lack math- U.S. teachers believe that their lessons ematically rich content suggest that our are already implementing the reform rec- students have less opportunity to learn ommendations, but the findings de- challenging mathematics than their scribed so far in this chapter suggest that counterparts in Germany and Japan. their lessons are not. When asked to evaluate to what degree the videotaped lesson was in accord with current ideas TO WHAT EXTENT ARE THE about teaching and learning mathemat- RECOMMENDATIONS OF THE ics, almost 75 percent of the teachers MATHEMATICS REFORM respond either “a lot” or “a fair amount.” MOVEMENT BEING IMPLEMENTED? This discrepancy between teachers’ beliefs and the TIMSS findings leads us A great deal of effort has been in- to wonder how teachers themselves un- vested in the reform of mathematics derstand the key goals of the reform teaching in the U.S. in recent years. movement, and apply them in the class- There is considerable agreement room. among experts about what good instruction should look like. The main Teachers in the study were asked to goal of the reform is to create class- describe which aspects of the video- rooms in which students are chal- taped lesson exemplified current lenged to think deeply about math- ideas about teaching and learning ematics and science, by discovering, mathematics. Most U.S. teachers’ understanding and applying con- answers fall into one of three catego- cepts in new situations. For many ries: years, Japanese mathematics educa- ■ Hands-on, real-world math - 38 tors have closely studied U.S. edu- percent of the teachers mentioned cation reform recommendations, and lesson activities that apply math attempted to implement these and to daily life, such as temperature other ideas in their own country. in Alaska, or that use a physical representation of a mathematical Has the message about mathemat- concept, such as geometric blocks. ics reform penetrated to U.S. classrooms? TIMSS data suggest that it ■ Cooperative learning - 31 percent is beginning to, but still only in limited of the teachers mentioned the use ways. Ninety-five percent of U.S. of peer tutoring, “study buddies,” or teachers stated that they were either math discussion groups. “very aware” or “somewhat aware” of ■ Focus on thinking - 19 percent of current ideas about teaching and the teachers mentioned focusing learning mathematics. When asked to on conceptual thinking about list titles of books they read to stay in- math in preference to computational formed about current ideas, one third skills, or mention focusing on prob- of U.S. teachers wrote down the names lem solving. of two important documents by the Na- tional Council of Teachers of Mathemat- Over 80 percent of the teachers in the ics, Curriculum and Evaluation Standards study referred to something other than

46 a focus on thinking, which is the central TIMSS test. However, until more stud- message of the mathematics reform ies of other high-scoring nations are car- movement. The majority of the teach- ried out, we cannot be sure that there is ers cited examples of hands-on math or a relationship between Japan’s high cooperative learning, which are tech- scores and its style of teaching. niques included among the reform recommendations. However, these techniques can be used either with or with- WHAT DO INITIAL FINDINGS out engaging students in real mathemati- SHOW ABOUT SCIENCE TEACHING? cal thinking. In fact, the videotape study observed many examples of these tech- TIMSS provides less data about science niques being conducted in the absence teaching than about mathematics teach- of high-quality mathematical content. ing, because the videotape study was conducted only in mathematics. How- These findings suggest that the in- ever, the TIMSS teacher and student structional habits and attitudes of questionnaires included some items U.S. mathematics teachers are only about instructional practices which help beginning to change in the direction us understand something about the of implementation of mathematics teaching of science in Germany, Japan, reform recommendations. Teachers’ and the U.S.. The questionnaire data implementation of the reform still has only begun to be analyzed, and more concentrates on isolated techniques analyses will soon be completed. Pre- rather than the central message, liminary analyses suggest that U.S. sci- which is to focus lessons on high-level ence teaching may resemble mathemat- mathematical thought. The finding ics teaching in some respects, and differ that almost 20 percent of the teachers in others. Therefore, one should not as- believed that they had implemented this sume that the videotape findings in focus on mathematical thinking, despite mathematics apply to science or to other experts’ judgments that a high-quality subjects. sequence of mathematical ideas was virtually absent in their lessons, suggests Taken together, the data suggest that that teachers may not yet understand the instruction in typical U.S. math- what the reform movement means by ematics classes is not of as high a this term. quality as that in other countries. Next, we turn to the TIMSS findings The videotape study found that, in concerning the teachers themselves. many ways, Japanese teaching resembled Do the daily working lives of U.S. the recommendations of the U.S. reform teachers provide as much support for movement more closely than did Ameri- their instructional activities as those can teaching. Japan also scored among of other countries? the top nations in the world on the

47 CHAPTER 4 : TEACHERS’ LIVES

KEY POINTS:

Unlike new U.S. teachers, new Japanese and German teachers receive long-term structured apprenticeships in their profession.

Japanese teachers have more opportunities to discuss teaching-related issues than do U.S. teachers.

U.S. teachers have more college education than their colleagues in all but a few TIMSS countries.

Student diversity and poor discipline are challenges not only for U.S. teachers, but for their German colleagues as well.

48 Hoping to improve U.S. classroom in- exceeded by only four other TIMSS struction, many policy makers have rec- countries. In Japan, few teachers had ommended improvements in various as- more than a Bachelors’ degree with pects of the U.S. teacher education sys- teacher training. In Germany, teachers tem. Experts agree that both the qual- complete 13 years of primary and sec- ity of the college preparation prospec- ondary school, followed by about six tive teachers receive as well as the qual- years of study at the university, after ity of the in-service training existing which they write a thesis and pass an teachers receive are important. How- examination to receive a degree consid- ever, each year, the percentage of newly- ered equivalent to a U.S. masters’ de- hired teachers is comparatively small in gree. relation to the size of the existing teaching force. Therefore, many experts agree Spending many years in college, how- that, in the short run, the quickest way ever, does not necessarily result in to improve students’ learning opportu- teachers who are experts in their fields. nities is to improve the instruction pro- Many U.S. policy makers consider it im- vided by existing teachers. portant for mathematics and science teachers to have a strong college back- To better understand how the charac- ground in those subjects. TIMSS, how- teristics of teachers’ daily lives may or ever, was unable to collect information may not contribute to high-quality on this topic due to the great variety of teaching, a team of twelve bilingual re- ways in which university training in searchers each spent three months in mathematics and science is organized in German, Japanese, or U.S. schools, ob- the participating countries. serving and interviewing teachers, principals, and students. This activity was Japanese and German teachers en- carried out as a supplement to the U.S. joy the security of the benefits and TIMSS effort. As this chapter will de- tenure which come from their status scribe, researchers found important dif- as civil servants. As civil servants, ferences between U.S. teachers’ oppor- their jobs are highly protected, and tunities for professional learning and im- they are comparatively free from con- provement of their teaching, and the op- cerns about labor-force downsizing portunities of their Japanese and Ger- or termination for incompetence. man counterparts. The typical teacher of U.S. eighth- grade math and science students was a woman in her forties, with about WHO TEACHES MATHEMATICS 15 years of prior teaching experience. AND SCIENCE? Forties was the norm for most of the other TIMSS countries. The typical U.S. teachers report that they have teacher of German students was a spent more years in college than man nearly fifty, who had been teach- teachers in all but a few of the 41 ing for about 19 years; and the typi- TIMSS countries. Nearly half of the cal teacher of Japanese students was teachers of U.S. eighth-graders had a a man in his late thirties, who had masters’ degree, a proportion which was been teaching for 14 years.

49 HOW DO TEACHERS SPEND The rhythm of U.S. and Japanese teach- THEIR TIME? ers’ daily school life was more similar than for their German colleagues. Ob- Teachers of the U.S. and German servations of U.S. teachers showed that eighth-grade students teach more they usually were at school around eight classes per week than Japanese teach- hours a day. They were expected to be ers. Questionnaires asked teachers to in the building during school hours, al- report the number of periods they teach though many came earlier, or stayed each week. Mathematics teachers in later. Japanese teachers were usually at the U.S. most commonly reported school around nine hours a day. They teaching 26 periods per week. Ger- were expected to be at school from the man teachers reported teaching 24, time it started in the morning until about and Japanese teachers reported 4:00 or 5:00, when student club activi- teaching 16 periods. Science teach- ties end. Many worked later on some ers in the U.S. and Germany most evenings. Japanese schools also were in often reported teaching 25 periods per session for a half day two Saturdays per week, and Japanese science teachers 18. month. Most mathematics teachers in all three countries taught few periods outside of German teachers of eighth-grade stu- their subject, and the same was true of dents spent the shortest amount of time science teachers. at school. The hours during which they were in the building usually varied from In addition to teaching, U.S. and Japa- day to day, depending on their teaching nese teachers are formally scheduled to schedule. During periods when they perform considerable additional duties were not scheduled to teach, teachers during the school day. In the U.S., often were not at school and felt free to teachers reported that these additional come and go from the school much as responsibilities are primarily in student college professors do in the United supervision and lesson planning. In Ja- States. Most returned home when pan, the time was roughly balanced be- school was over around 1:30, ate their tween student counseling, administra- lunch at home, and planned lessons and tive duties, and lesson planning. Most reviewed student work during the after- German teachers were scheduled for noon and evening. very few hours of non-teaching tasks at school, and they did their lesson plan- U.S. and German teachers do not have ning at home. the rich informal opportunities to learn from each other and to share Eighth-grade math and science class questions about teaching-related is- sizes in the U.S. and Germany were sues that are enjoyed by their Japa- about the same, averaging 24 to 25 stu- nese colleagues. Japanese schools are dents per class. Japanese math and sci- designed with one very large teachers’ ence classes were much larger, aver- room, in which all teachers have their aging 37 students. main desks, and the seating is arranged so that all teachers from a particular

50 grade or subject sit near each other. vation, then progresses to assisted teach- When they were not actually instruct- ing, and finally to unassisted teaching ing classes, teachers spent most of under the close direction of a mentor their time in this large room with their teacher. They also attend seminars in colleagues, providing many casual oppor- their subjects once or twice a week, and tunities each day to share advice, ideas their seminar instructor joins the men- and teaching materials. Japanese cul- tor in observing and evaluating the pro- tural norms expect junior teachers to spective teacher. At the end of the sec- query their older colleagues for teach- ond year, candidates take another state ing tips and rely on their advice. examination and apply for jobs. Place- ment is not guaranteed. Formal discussions between teachers were more frequent in Japan, as In Japan, after passing the teacher well. When asked how often they certification and employment selec- meet to discuss curriculum, 76 per- tion examinations, successful candi- cent of the teachers of the Japanese dates are hired by various prefec- TIMSS students reported “at least tures, which are similar to U.S. once a month,” compared to 60 per- states. New teachers then undergo cent of the U.S. and 44 percent of the intensive mentoring and training German teachers. during their first year on the job. New teachers’ first year includes at least 60 days of closely mentored teaching and HOW DO TEACHERS LEARN TO 30 days of further training at resource TEACH? centers run by the local and prefectural boards of education. Their teaching load U.S. teachers lack the long and care- is reduced to allow time for these activi- fully mentored introduction to teach- ties. As is typical of Japanese society, ing that Japanese and German teach- mentoring and assistance between jun- ers receive. In Germany this period of ior and senior teachers continues intensive training comes before being throughout teachers’ working lives. hired as a teacher. In Japan, it comes during the first year on the job. In all In comparison to the intensive on- three countries, prospective teachers first the-job training that German and take a mixture of courses in education Japanese teachers receive, U.S. and in academic subject areas leading teachers’ induction is less structured to graduation from college. After this, and comprehensive. Prospective U.S. however, their experiences diverge teachers typically spend 12 weeks or sharply. less in student teaching near the end of their undergraduate training. Af- In Germany, after passing a state ex- ter meeting state licensing require- amination at the end of college, pro- ments and being hired by a school spective teachers spend two years in district, the nature of the induction student teaching in a program resem- program varies by district, and may bling a traditional apprenticeship. include some type of in-service train- During the two years, prospective ing, and some mentoring by a more teachers have a reduced teaching experienced teacher. load that begins with classroom obser-

51 FIGURE 12 PERCENTAGE OF EIGHTH-GRADE MATHEMATICS TEACHERS REPORTING THAT VARIOUS CIRCUMSTANCES LIMIT THEIR TEACHING “QUITE A LOT” OR “A GREAT DEAL”

51 UNINTERESTED STUDENTS 43 37

44 DIFFERENT ACADEMIC 55 ABILITIES 63

29 HIGH STUDENT/ TEACHER RATIO 38

39 DISRUPTIVE STUDENTS 40

NotN/A asked

31 UNINTERESTED PARENTS 19

Not asked

29 LOW STUDENT MORALE 15

Not asked SOURCE: Third International Mathematics and Sci- U.S. ence Study: Unpub- 7 THREATS TO GERMANY lished Tabulations, U.S., German, and PERSONAL/ 8 JAPAN STUDENT SAFETY Japanese teacher surveys, Westat, 1996. Not asked

0 10 20 30 40 50 60 70

52 WHAT CHALLENGES DO Severe discipline problems and TEACHERS FACE? threats to student and teacher safety are neither widespread nor unique to Although teaching students is their job, the U.S., despite stories in the popular dealing with students can be teachers’ media that sometimes give the impres- greatest challenge. During interviews, sion that these problems do not exist in teachers in all three countries fre- other countries. An approximately equal quently described student diversity as and small number of German and U.S. a challenge. Diversity takes different eighth-grade teachers reported feeling forms in each country, however. U.S. that threats to themselves or their stu- teachers referred primarily to differences dents’ safety limited their effectiveness in American students’ social, economic, as teachers. Most teachers, however, or ethnic background, or to the chal- never experience such serious problems. lenges of dealing with non-English- Seventy-six percent of the U.S. and 65 speaking students. German teachers re- percent of German teachers reported ferred to differences in ethnic back- that threats to their own or students’ ground, language, and national origin be- safety did not limit their effectiveness tween the children of German citizens at all. TIMSS researchers who ob- and their country’s foreign workers. served and interviewed teachers in Japanese teachers referred to the their schools reported that, in both wide differences in academic ability countries, the schools with such se- within each classroom, which arise from rious problems were generally in their nation’s policy of not separating poorer areas of the city. students by ability in any way until high school, and not retaining low-perform- Science teachers in all three coun- ing students in grade. tries reported hindrances similar to those of their mathematics col- What circumstances do teachers in the leagues, except that they added short- three countries believe limit their effec- ages of demonstration and instructional tiveness? TIMSS questionnaires asked equipment to the circumstances which teachers to rate the extent to which vari- limit the effectiveness of their teaching. ous factors limited their ability to teach. Figure 12 on page 52 shows the results. Students themselves reported somewhat more discipline problems than Uninterested students and a wide their teachers, possibly because chil- range of academic abilities challenge dren often do not report all incidents teachers in all three countries. Over to school authorities. About 25 per- a third of U.S. and German eighth-grade cent of the eighth-graders in both teachers also felt that disruptive stu- Germany and the U.S. reported on dents limited their effectiveness as the questionniares that, during the teachers. The Japanese chose not to past month, they had been afraid that include any questionnaire items re- another student might hurt them. lating to discipline or morale prob- About 40 percent in each country lems in their schools. said that one of their friends had been hurt by another student. Theft was

53 FIGURE 13 DISCIPLINE PROBLEMS EIGHTH-GRADE PRINCIPALS DEAL WITH ON A DAILY BASIS

58.0 CLASSROOM DISTURBANCES 68.0

53.0 TARDINESS 33.0

INTIMIDATION OR 21.0 VERBAL ABUSE OF STUDENTS 18.0

PHYSICAL INJURY 7.0 TO OTHER STUDENTS 2.0

INTIMIDATION OR VERBAL ABUSE 8.0 OF TEACHERS/ 1.0 STAFF

PHYSICAL INJURY 3.0 TO TEACHERS/ STAFF 0.0

6.0 VANDALISM 5.0

POSSESSION 2.0 OF WEAPONS 3.0

INAPPROPRIATE 2.0 SEXUAL BEHAVIOR 5.0 U.S. GERMANY USE OF 2.0 ILLEGAL 2.0 DRUGS 0 10 20 30 40 50 60 70 80

SOURCE: Third International Mathematics and Science Study; unpublished tabulations, U.S., German, and Japanese School Surveys; Westat, 1996.

54 more common in the U.S. than Ger- staff were rare. Use of illegal drugs and many. Fifty-eight percent of U.S. stu- possession of weapons was reported as dents but only 32 percent of German a daily problem by only about 2 percent students said that one of their friends of the U.S. and German principals. Over had something stolen during the past 90 percent of principals reported that month. Skipping classes was more com- they and their staff dealt with these mon in Germany, with 66 percent of Ger- problems rarely or never.. man students reporting that one of their friends had skipped class during the past Teachers in all three countries found month, compared with 50 percent in the dealing with student diversity to be a U.S. challenge to their effectiveness. Many German teachers also experienced prob- Figure 13 on page 54 shows the percent- lems with student misbehavior. Many age of U.S. and German principals who teachers in all three countries believed reported that they dealt with various their effectiveness was limited by the kinds of discipline problems on a daily range of student abilities represented in basis. Principals in both countries re- their classes, and also by disruptive and sponded that their most common disci- disinterested students. The next chap- pline problems were classroom distur- ter turns to the questions of how nations bances, tardiness, and intimidation or deal with student ability differences, as verbal abuse of students by other stu- well as the supports and incentives of- dents. More serious problems such as fered to students in their academic en- physical injury of students, teachers, or deavors.

55 CHAPTER 5 : STUDENTS’ LIVES

KEY POINTS:

Eighth-grade students of different abilities are typically divided into different classrooms in the U.S., and different schools in Germany. In Japan, no ability grouping is practiced.

In the U.S. students in higher-level mathematics classes study different material than students in lower-level classes. In Germany and Japan, all students study the same material, although in Germany, lower-level classes study it less deeply and rigorously.

Japanese eighth-graders are preparing for a high-stakes examination to enter high school at the end of ninth grade.

U.S. teachers assign more homework and spend more class time discussing it than teachers in Germany and Japan. U.S. students report about the same amount of out-of-school math and science study as their Japanese and German counterparts.

Heavy TV watching is as common among U.S. eighth graders as it is among their Japanese counterparts.

56 On the surface, the lives of eighth grad- JAPAN ers in most TIMSS countries are fairly similar. School and family occupy the Japanese public schools offer a single biggest portions, with friends, TV, home- curriculum for all students through work, clubs, and fun added around the the end of 9th grade. Students in el- side. Yet below the surface, the way in ementary and junior high schools are vir- which societies choose to structure the tually never tracked or grouped by aca- schooling process gives rise to different demic ability. There is a widespread be- opportunities and expectations for lief that, to be fair to all students, the young people. The motivators, supports, nine years of compulsory education and obstacles to study in each country must offer the same nationally deter- are outgrowths of the choices provided mined curriculum to all, regardless of in- by society and schools. In each coun- dividual differences in motivation or try, the expectations which adult soci- ability. Until the end of ninth grade, ety sets for young people form a frame- there are no gateway exams, and all stu- work within which students organize dents are promoted whether or not they their lives. understand the material. Students who are overly or insufficiently challenged by classroom assignments may receive extra help after school from a teacher, or WHAT DOES THE SYSTEM REQUIRE their parents may pay to enroll them in OF STUDENTS? a juku, which is a private after-school class. In Japan, a substantial amount of Some U.S. education policy makers have remedial and enrichment instruction is looked admiringly at other nations provided by the private sector. which use periodic gateway examinations to control student access to the In mathematics, all eighth-grade Japa- next level of education. Such high-stakes nese students receive a curriculum tests are believed to encourage students heavily focused on algebra and geom- to study hard. The German and Japa- etry. Review of arithmetic is not in- nese systems are frequently cited as excluded in the official curriculum goals amples by the proponents of such prac- and textbooks. TIMSS observers noted tices. TIMSS allows us to compare the that there are differences in students’ pathways through schooling in these two ability to keep up with the curriculum countries to those of our own, to under- within each classroom, and also between stand how the expectations built into the schools where students come from fami- system motivate students of different lies with predominantly high or low eco- ability levels. nomic backgrounds. However, the Japa- nese system is designed such that teachers throughout the country strive to meet similar standards for presentation of content, while allowing almost unlim- ited variation in the standards of performance attained by students.

57 At the end of ninth grade, virtually GERMANY all Japanese students continue on to high school. Before they do, however, Various exceptions and experiments all must take the high school entrance notwithstanding, the German school exam. This examination covers the five system basically sorts students into core subjects, including mathematics one of three types of schools at the and science. Scores on the examination end of the fourth grade of elementary serve as a gateway which divides stu- school. This is accomplished through a dents into high, medium, and low-level system of gateway examinations and high schools on the basis of each ability grouping which differs consider- student’s scores on the exam and prior ably from the Japanese. Most German academic performance. The best of the students attend one of three types of graduating ninth-graders are accepted at schools: the best academic high schools in each city, which prepare students for applica- ■ Gymnasium, which provides a de- tion to the best universities. The slow- manding, academic curriculum est students are accepted only by the through grade 13 and leads to the lesser-ranked commercial or vocational Abitur exit examination and univer- high schools, which prepare graduates sity study. to enter the labor force. Students and parents clearly understand the conse- ■ Realschule which provides a moder- quence of this examination at the end ately-paced curriculum ending at of ninth grade for future career and life grade 10 and leads to a school-leav- choices. Japanese students say that the ing certificate and vocational train- examination motivates them to study ing or further study at a Gymnasium. harder during their junior high school years. For the majority of Japanese stu- ■ Hauptschule, which provides practi- dents, this is the only high-stakes exam cally-oriented instruction ending at they will experience. grade 9 and leads to a school leaving certificate and vocational training or Once Japanese students enter high employment. Immigrant and non- school, they are again promoted each German students are over-repre- year, until they graduate. Most students sented in the Hauptschule. then enter the labor force or vocational The gateway into one of these schools training. Approximately one third of the is controlled by teacher recommenda- high school graduates decide to apply tions at the end of fourth grade. Par- to a university or two-year college, most ents can, and frequently do, override of which require an entrance examina- teacher recommendations if they believe tion. Competition on the entrance ex- that their child deserves to be placed in aminations for prestigious universities is a higher track. If the student is unable intense, although some lower-ranked col- to keep up with his classmates, however, leges will accept most high-school gradu- he or she will be retained in grade and ates who apply. after repeated failure will be returned to

58 the next lower level of schooling. Most repeat the grade, or may be moved to a German parents and teachers are rela- less demanding school type. Principals tively comfortable with the fairness of reported that 5 percent of students were this system, because they believe that it required to repeat grade eight. Most stu- allows each child an education best dents finishing the Hauptschule at the suited to his or her abilities, interests and end of grade 9, or Realschule at the end future career. However, there is a sub- of grade 10 receive a diploma, and most stantial current of opinion within Ger- states do not require an exit exam. many which would prefer to delay the About 10 percent of the students receive sorting of students into different school only a school-leaving certificate instead types until later in the student’s life, and of a diploma. Approximately one-third to make it easier for students to change of German students are enrolled in a upward to a higher school type. Most Gymnasium, and about a quarter of these recent policy reforms have made small end their studies before taking the Abitur changes to modify the system in this di- examination at the end of 13th grade. rection. Very few students who sit for the Abitur fail it, although those with a lower score Classes in grades 5-9 basically cover the may not be able to enter their chosen same content in all three types of Ger- university or field of study. man schools, although there is considerable difference in the depth and rigor of instruction between the three school UNITED STATES types. Typically, Gymnasium students receive a theoretical approach, and It is more difficult to generalize about Hauptschule students receive a practical the United States, because practices dif- approach to the same content. In fer among the thousands of school dis- eighth-grade mathematics, the German tricts in the country. Generally speak- curriculum focuses mostly on Geometry ing, however, within-class grouping or in- and Algebra for all three types of schools, dividuation of instruction is fairly com- with some mixture of other topics. mon in elementary schools in the subjects of reading and mathematics. In Within most schools, eighth graders all middle schools and high schools, stu- follow the same course of study in math dents are frequently grouped by abil- and science, regardless of their ability ity into different mathematics classes. level. Seventy-five percent of the schools In the U.S., 80 percent of principals of reported that they provide only one eighth graders reported that they pro- course of study in mathematics, and 90 vided different ability-based classes in percent provide only one course in sci- mathematics, but only 17 percent re- ence. Generally speaking, the German ported this in science. Course content system separates students into different and textbooks usually differ between the ability levels primarily between, rather higher and lower-level classes. In the than within, schools. eighth grade,

In Germany, students who have not learned the material may be required to

59 lower-level classes typically focus on systems in the three countries. Japan is a review of arithmetic and other ba- the only one of the three countries sic skills with a small amount of al- which requires a high-stakes entrance gebra. Higher-level classes focus more examination for all students. Math- heavily on algebra, with a small ematics and science are included on this amount of geometry. examination, and Japanese eighth-graders are therefore likely to be studying In the U.S., educational expectations these subjects harder than usual in and teaching standards can also differ preparation. Methods of sorting stu- substantially between communities, dents by ability into schools and classes based on a neighborhood’s economic differ among the three countries, but status and parental expectations for their both Germany and Japan teach alge- children’s futures. Minority students are bra and geometry to all of their eighth- over-represented in lower-level classes grade students, although the level of and in schools in poorer areas. rigor may differ by track. In contrast, in the U.S. a heavy focus on algebra There are various procedures for deal- is usually reserved for students in the ing with students who teachers judge higher tracks, and few U.S. eighth- have not learned the course material. graders in any track study much ge- They may be promoted anyway, retained ometry. in grade, moved to a lower-tracked class, or given remedial assistance. Principals In all three countries, the standards of reported that 4 percent of the students performance for students at each grade in their schools were required to repeat level are set in such a way that almost grade eight. all students are passed from one grade to the next, and all who complete sec- Generally speaking, the U.S. system does ondary education can obtain some type not have high-stakes gateway examina- of secondary school diploma, regardless tions which regulate entrance to further of their level of academic ability. schooling before the end of twelfth grade. Seventeen states currently have an exit examination as a requirement for high-school graduation. In most cases, HOW DO STUDENTS SPEND THEIR this is a minimum-competency test. Stu- TIME DURING SCHOOL? dents may take the test several times if necessary, and few students repeatedly fail. Scores on college entrance exami- UNITED STATES nations such as the SAT and ACT are U.S. students attend school approxi- given considerable weight by most se- mately 180 days per year, five days per lective universities, although non-selec- week. Each day, school usually runs from tive schools may not require them at all. about 8:00 in the morning until mid- afternoon, with a lunch break and five This section has examined the learning to seven-minute breaks between classes. expectations embedded in the school

60 Schools vary in how they organize stu- Most German schools offer few extra- dents. Middle schools commonly in- curricular activities. Schools visited by clude either grades 7-9, or 6-8, although TIMSS observers offered mostly sports, variations exist. In some schools, the arts, and student government. Student student body is subdivided into “houses” participation was low, and some clubs or “blocks” which include several classes rarely met. Six percent of German stu- of students and a single group of teach- dents said that they participate in a math ers, to strengthen continuity in student- or science club each week. Over half of teacher and student-student relation- all German students under the age of ships. In other schools, students change 15 are involved in organized sports, but teachers and classmates at the end of these are sponsored by a national each period. organization’s local sports clubs rather than the school. Most U.S. schools offer a variety of teacher-led after-school activities, including sports, music, art, theater, and aca- JAPAN demic clubs. The range of after-school activities varies by school and often re- Japanese schools are in session 220 days flects the district’s and school’s resources per year, five days per week, and two Sat- and socioeconomic status. Participation urday mornings per month. School usu- in clubs is voluntary, and students can ally starts at 8:00 in the morning and participate in more than one activity, as ends in the middle of the afternoon, with some are seasonal or do not meet every a lunch break, 5 to 15 minute breaks day. Ten percent of U.S. students said between various periods, and a that they participate in some type of homeroom meeting at the beginning and math or science club each week. end of each day. The number of classes per day is frequently reduced for special seasonal events, school-wide meetings, GERMANY and other activities. Junior high schools include grades 7-9. Students in a given German students attend school approxi- class remain together throughout the mately 188 days per year. School usu- day, and a different teacher for each sub- ally starts around 7:45 in the morning, ject comes to the students’ classroom. and ends around 1:15, with 10 to 25 minute breaks between classes. There Extracurricular or “club” activities are a is no lunch period, and most students very important part of Japanese eighth- return home for lunch. Gymnasium usu- graders’ lives, and well over half of all ally include students from grades 5-13, students participate. Clubs meet daily Realschule grades 5-10, and Hauptschule throughout the year from the time that grades 5-9. Eighth-grade students re- classes are over until about 5:00 or 6:00. main together throughout the day, with Four percent of Japanese students re- teachers changing classrooms. Classes ported participating in a math or science are usually kept together for several years club. and develop a strong sense of unity.

61 In contrast to their German and U.S. HOW MUCH STUDY DO counterparts, Japanese junior-high STUDENTS DO AFTER SCHOOL? school students are required to wear uni- forms to school, and must follow a strict Study at home is not the same as home- dress code. Regular uniform inspections work. Ideally, students would be self- chastise such deviations as non-regula- motivated to study mathematics and sci- tion belts, shoes, hairstyles, jewelry, and ence more than the minimum required non-regulation book bags. The students by homework assignments. The degree themselves play a major role in the en- to which this actually happens depends forcement of school rules and discipline. on the individual student, and the de- Between students, there is a complicated gree to which the culture encourages or senior-junior system of deference and requires eighth-graders to take respon- behavior training. Younger students sibility for their own learning. speak to students in upper grades using the respectful term sempai (upper-class Interviews with students about their man/woman). Particularly within the daily lives found that, in all three coun- clubs, upper-class students are in charge tries, most students tended to put in ex- of overseeing the younger students. tra non-assigned study before examinations and relax after they were finished. In Germany and the U.S., the only tests

FIGURE 14 PERCENT OF MATHEMATICS AND SCIENCE TEACHERS WHO ASSIGN HOMEWORK 3 TO 5 TIMES PER WEEK 100 90 86 MATHEMATICS 80 75 SCIENCE 70 60 50 48 40 30 21 20 12 10 4 0 U.S. GERMANY JAPAN

SOURCE: Third International Mathematics and Science Study; unpublished tabulations, U.S., German, and Japanese National Surveys; Westat, 1996.

62 with some consequences for students’ U.S. and German teachers not only as- academic lives were periodic teacher-pre- sign more homework than Japanese, pared in-class examinations. There were but they also spent more class time broad similarities across countries in stu- talking about or doing it. Time spent dents’ strategies of study for these ex- on assigning, working on, or sharing aminations. High-achieving students de- homework occupied 11 percent of U.S. scribed doing extra hours of non-as- and 8 percent of German lessons, in signed review and preparation, while this comparison to 2 percent of Japanese was much less common among low lessons. Furthermore, most U.S. teach- achievers. In Japan, consciousness about ers reported that they counted home- the examinations at the end of ninth work toward student grades, whereas grade caused all eighth graders to be this practice was not common in Ger- mindful of the need for extra personal many and Japan. It was only in the U.S. study and preparation, although high that some teachers allocated class time achievers were more likely to translate for students to begin their homework in this into substantial home study. class.

Most Americans believe that homework The picture changes, however, when stu- is an important part of the learning pro- dents themselves were asked how much cess. Some have recommended assign- time they spend studying math and sci- ment of more homework as a means of ence. On average, Japanese, German, improving mathematics achievement. It and U.S. students reported that they is frequently assumed that teachers in spent about the same amount of time high-achieving countries assign more each day — between 30 minutes and homework than do U.S. teachers. an hour — studying mathematics outside of school, and about the same However, TIMSS found that Japanese amount studying science. These ques- teachers actually assigned less home- tionnaire findings are in line with what work than U.S. and German teachers. interviewers found when they spoke The teacher questionnaire results and with eighth graders in each country videotapes of classroom practices both about their study habits. agree on this finding. Figure 14 on page 62 shows that 86 percent of U.S. math- Between 30 minutes and an hour of ematics and 75 percent of German after-school study per night is an aver- teachers assigned homework 3 to 5 times age in each country. Of course there per week, in comparison to 21 percent were wide differences between students of Japanese teachers. When asked about everywhere in how willing they were the amount of homework they assign, to complete assignments or go beyond U.S. and German math teachers’ most them in extra personal study. Some common response was about thirty min- German, Japanese, and U.S. teachers utes or less, three or more times per noted that low-achieving students, par- week. Japanese teachers typically as- ticularly those from troubled family signed the same amount, but once or twice per week.

63 backgrounds were less likely to complete dents study in more depth to prepare assignments, either because they lacked for entrance examinations. Parents must the motivation, or did not have a family pay to send their children to these pri- environment which was conducive to vate classes, which are run by compa- home study. In contrast, some high- nies or neighborhood tutors. Research- achieving students in each country en- ers reported that some mothers take an gaged in extra study beyond what was extra job to provide the tuition. Al- assigned. though the purpose of juku is academic, students enjoy attending them, because If Japanese teachers assigned less home- they are able to spend time with their work than German and U.S. teachers, friends walking or riding subway trains but Japanese students reported that they to and from the classes. Sixty-four per- studied about as much as their counter- cent of Japanese eighth graders re- parts in these countries, how were typi- ported attending weekly extra lessons cal Japanese students motivated and in math, and 41 percent in science. supported in this extra study? Research- Most students attend juku one or two ers who observed and interviewed in hours per week. Attendance drops off Japanese schools and homes reported substantially once high school entrance that parents, teachers, and friends en- examinations are completed. Other couraged students to study hard during types of non-academic after-school their eighth and ninth grade years in classes, such as music or marital arts, preparation for the high school entrance were also popular among Japanese stu- examinations. Students are believed to dents. have considerable personal responsibil- ity for this process. Some popular teen Japanese experts report that instruction magazines even run articles on how to in mathematics juku focuses more on re- devise a personal study and review plan. view and practice of basic skills than is Japanese students described a combina- typical of Japanese classrooms. This as- tion of peer support and competition sists slower students who need review that encouraged them to study harder of prior material, and provides all stu- during these years. For students who dents extra practice with concepts enter a commercial or vocational high learned but not drilled upon in class. school, however, extra study tends to fall Although more systematic study of juku off again after entrance to high school. instruction is needed, the hypothesis might be entertained that Japanese stu- Another important source of outside as- dents benefit from the different but sistance for Japanese students is the juku, complementary nature of juku and class- which are private after-school classes of- room instruction. fered in a variety of subjects to help slower students catch up, or faster stu-

64 FIGURE 15 PERCENT OF EIGHTH-GRADERS SPENDING 3 OR WHAT DO STUDENTS THINK MORE HOURS IN VARIOUS AFTER-SCHOOL ABOUT MATHEMATICS AND ACTIVITIES ON A NORMAL SCHOOL DAY SCIENCE?

At least half the students in Germany, 38 WATCHING Japan, and the U.S. reported that they TELEVISION 39 like math and science. In the U.S., OR VIDEO boys and girls were equally positive, but 22 German and Japanese girls were less positive than boys in those countries. 37 PLAYING How much students like math and sci- WITH 26 FRIENDS ence is a different question. Students 62 in all three countries were more in- clined to agree that that it was important to have time to have fun than 31 ENGAGING that to do well in mathematics and IN SPORTS 12 science. More students in the U.S. also agreed that it was important to do well 20 in sports than to do well in math and science. In Germany and Japan, how- 6 ever, fewer students considered it impor- READING A tant to do well in sports than in math- BOOK 5 ematics. FOR FUN 5 Japanese policy makers are currently discussing an emerging social phenom- 3 enon they term risu kirai, or “dislike of STUDYING 1 mathematics and science.” Although MATH much discussed among Japanese ex- 1 perts, it is not clear how widespread this phenomenon is in Japan. About 10 percent of Japanese students re- 2 ported that they disliked mathematics STUDYING 1 “a lot,” which was comparable to the SCIENCE number of U.S. students who reported 1 strongly disliking the subject. Inter- views with Japanese students who dis- U.S. 5 liked the subject suggest that they dis- STUDYING JAPAN liked it because they saw it as difficult OTHER 3 GERMANY SUBJECTS and uninteresting. Japanese teachers 2 speculated that many of these students 0 10 20 30 40 50 60 70 may have fallen behind in earlier grades and never caught up. The teachers SOURCE: Third International Mathematics and Science Study: Un- thought that the demanding pace of published Tabulations, U.S., German, and Japanese Surveys, Westat, 1996.

65 the curriculum and the need to keep in- Students who watched a lot of televi- struction focused on the material which sion each day after school were fairly will be covered on the high school en- common in all three countries, especially trance examination caused students to the U.S. and Japan. After-school sports fall behind. were more popular in the U.S than in Germany or Japan. Almost one third of Most Japanese students experience U.S. eighth-graders reported spending mathematics and science as difficult. three hours per day engaged in sports Eighty-seven percent disagreed with the activities. In Germany, friends were statement “math is an easy subject,” and more popular than television. Two-thirds 85 percent disagreed with a similar state- of German students spent at least three ment in science. About half of U.S. stu- hours per day playing with friends, pos- dents on the other hand, reported that sibly because German schools finish be- math and science are easy. Given the fore lunch, and students have more time findings reported in Chapter 3 that the to spend with their friends in the after- U.S. mathematics curriculum focuses on noon. Very few students in any of the easier topics, and that classroom activi- three countries spent extended periods ties are based mostly on routine proce- of time reading books for fun or study- dures rather than conceptual thinking, ing school subjects. the hypothesis might be entertained that U.S. students’ classroom experiences, at The priorities that nations assign to least in mathematics, lead them to be- schooling are evident in the opportuni- lieve that these subjects are easy. ties provided for students outside of school. Japan tries to encourage eighth- graders to focus primarily on school, family, and study. In contrast to U.S. WHAT DO STUDENTS DO AFTER and German schools, Japanese schools SCHOOL BESIDES STUDY? set and enforce policies for behavior off school grounds. Examples include poli- What other choices and opportunities cies regarding curfews; clothing to be do societies offer their eighth-graders be- worn in public; use of bicycle helmets: sides focus on school and study? The and prohibitions against entering game way in which societies structure the arcades, dating, employment, smoking, choices available to young people shows and alcohol. In some towns, teachers something about the priority assigned and parents check shopping malls, parks, to schooling and the society’s investment and other areas where students are likely in education. to congregate to monitor student compliance with the rules. These policies Figure 15 on page 65 shows that eighth- may contribute to Japanese students’ graders in all three countries were reports that they spent less time with more likely to spend extended periods their friends than German and U.S. after school watching television or vid- teenagers. eos, playing with friends, or engaging in sports than taking part in more aca- In the U.S. and Germany, working at a demically-related activities. paid job was not uncommon even for

66 eighth graders. About a quarter of all its schooling process, and the expecta- students in these countries reported that tions that it sets for students provide they worked at a paid job before or af- different motivators, supports, and dis- ter school at least an hour per week. In tractions from study. Considering the Japan, this percentage was 4 percent. choices that other nations have made in this regard may help us to better under- In summary, eighth-graders’ lives in Ger- stand our own. many, Japan, and the U.S. share broad similarities in their focus on school, friends, TV, and sports. However, the way in which each society has designed

67 C O N C L U S I O N S

KEY POINTS:

No single factor can be considered to influence student performance in isolation from other factors. There are no single answers to complex questions.

The content of U.S. eighth-grade mathematics classes is not as challenging as that of other countries, and topic coverage is not as focused.

Most U.S. mathematics teachers report familiarity with reform recommendations, although only a few apply the key points in their classrooms.

Evidence suggests that U.S. teachers do not receive as much practical training and daily support as their German and Japanese colleagues.

68 This report has presented highlights both subjects. Particularly in mathemat- from initial analyses of U.S. eighth-grad- ics, our students lag far behind top-rank- ers in international perspective. These ing countries. Compared to our goal findings lightly sketch only a corner of of excellence among nations, we are the entire picture of U.S. performance not where we aim to be. in mathematics and science which will be painted over the next years as fur- However, we are on a par with many ther analysis of the eighth-grade data is of our international trading partners. carried out and findings from grades four Our students stand not far from the in- and twelve are added. ternational average: somewhat below in mathematics, and somewhat above in This section looks across the findings science. Our math scores are not sig- presented in the previous pages for in- nificantly different than those of Ger- sights into the key questions with which many and England. Our science scores the study started: How do our eighth- are not significantly different than those graders compare to their international of Germany, England, Canada, and Rus- counterparts? What have we learned sia. We rank near the middle of the 41 about mathematics achievement and the TIMSS countries, among other nations factors that may be associated with it? to whom we frequently compare our- What have we learned about science? selves. What have we learned about how our education system as a whole compares to that of other countries? WHAT HAVE WE LEARNED ABOUT Looking for insights into factors associ- MATHEMATICS? ated with student performance is com- Our eighth graders score below the in- plicated because achievement after eight ternational average in mathematics. Al- years of schooling and thirteen years of though international comparisons over life is the product of many different in- time are difficult, there does not appear fluences. Furthermore, education in our to have been much improvement dur- country is a vast system with many in- ing the past three decades in U.S. stu- terrelated parts. No single factor can dents’ international standing in this sub- be properly considered in isolation ject. The following factors may be asso- from others. Realizing that there are ciated with this performance: no single answers to complex questions, let us review the data. ■ The content of U.S. eighth-grade mathematics classes is not as challenging as that of other countries. WHERE DO WE STAND? U.S. eighth-grade curriculum and in- The U.S. is far from being among the struction both appear to be less chal top nations of the world in mathematics and science. We are far from this goal. Singapore, Korea, Japan, the Czech Re- public, and Hungary outperform us in

69 lenging than those in other countries. they are aware of current ideas about Concerning curriculum, topics cov- teaching and learning mathematics. ered in U.S. mathematics classrooms Most believe that the lessons they are at a seventh-grade level in com- teach exemplify elements of the rec- parison to other countries. Virtually ommendations. However, the way in all German and Japanese students which U.S. teachers understand and study algebra and geometry in the implement these recommendations eighth grade, while in the U.S., only suggests that they are focusing on iso- students in higher-level classes receive lated techniques rather than the cen- significant exposure to algebra, and tral message that teaching and learn- few students study geometry. ing should involve high-level mathematical thought. Our mathematics Concerning instruction, the content teachers’ typical goal is to teach stu- of U.S. classes requires less high-level dents how to do something, rather thought than classes in Germany and than how to think about and under- Japan. The sequence of mathemati- stand mathematical concepts. In a cal ideas used in lessons was judged variety of respects, Japanese math- to be of low quality in a majority of ematics teaching more closely re- U.S. classrooms, while this was less sembles the recommendations of the frequently the case in the other two U.S. reform movement. countries.

■ Topic coverage is not as focused in U.S. eighth-grade mathematics WHAT HAVE WE LEARNED ABOUT classes as in the classrooms of SCIENCE? other countries. U.S. eighth graders score above the in- In the U.S., curriculum is determined ternational average in science. In the at the state and local level, which is three previous international science as- atypical among TIMSS countries, sessments, the U.S. scored below the in- most of whom determine curriculum ternational average. Because compari- nationally. In all grades 1-8, the U.S. sons of different international assess- mathematics curriculum recom- ments over time are difficult, caution mends coverage of more topics than should be exercised in assuming that the international average. U.S. math- there has been significant improvement ematics lessons also include a greater in our international standing in science, number of topics and activities than but it is a possibility. those in Germany and Japan. This initial report contains less informa-

■ Most U.S. eighth-grade math tion about science than about math- teachers report familiarity with ematics because the questionnaire data reform recommendations, al- have not yet been fully analyzed, and though only a few apply the key the videotape study of classroom instruc- points in their classrooms. tion was conducted only in mathematics. Furthermore, because we are unable Ninety-five percent of U.S. eighth- to use multiple research methods to grade mathematics teachers say that

70 verify the science findings from differ- dents in Germany and Japan. U.S. ent perspectives, our findings are more teachers assign more homework, and tentative than for mathematics. spend more class time discussing it than teachers in those countries. Fuller description of eighth-grade science Outside of school, our students re- teachers’ instructional practices must port doing about as much math and await further questionnaire analysis. science-related homework and other study as German and Japanese students, although most Japanese eighth graders also attend after-school WHAT HAVE WE LEARNED ABOUT classes in mathematics for an hour U.S. EDUCATION AS A WHOLE? or two per week in preparation for the entrance exams to high school. TIMSS provides several insights about U.S. eighth-grade teachers and students, which are true of both mathematics and science education. QUESTIONS FOR FURTHER STUDY

■ Evidence suggests that U.S. teach- The initial findings described in this re- ers do not receive as much practi- port raise many important questions for cal training and daily support as further study. Some of these may be an- their German and Japanese col- swered through continued analysis of the leagues. eighth-grade data. Others must await the design of future international stud- In contrast to new German and Japa- ies. For this reason, TIMSS is an im- nese teachers, new U.S. teachers do portant national resource for secondary not receive a long-term structured ap- analysis and further research. Some ex- prenticeship in their profession. amples are: Once on the job, they have fewer formal and informal opportunities to ■ Why is our international standing discuss and share teaching-related is- lower in mathematics than in sci- sues and questions. Schools are man- ence? aged in such a way that lessons are frequently interrupted by loud- Deeper analysis of the TIMSS data speaker announcements or visitors at will help us to compare the curricu- the door. lum and instructional practices used in mathematics with those in science, ■ Our eighth-graders spend at least to better understand the similarities as much time studying mathemat- and differences. ics and science as students in Ger- many and Japan.

During school, our eighth graders spend more hours in mathematics and science classes per year than stu-

71 ■ How is student achievement re- THE NATION lated to curriculum coverage? TIMSS is not an answer book, but a mir- Comparison of the curriculum analy- ror through which we can see our own sis with achievement scores in the education system in international per- various content areas can illuminate spective. It helps us view with new eyes the degree to which our students’ per- those aspects of our system which we formance in algebra, earth science, may take for granted. Its findings make and other content areas is related to us think more deeply about the cultural curricular emphasis in these areas. assumptions and unconscious choices which form the underpinnings of our ■ Does mathematics teaching in society’s approach to schooling. We high performing countries re- come to understand our own system bet- semble the reform movement’s ter by comparing it to others. Careful recommendations? study of our country’s reflection in the mirror of international comparisons can The videotape study found that in provide information to assist educators, many ways, Japanese mathematics business leaders, teachers, and parents teaching resembles the recommenda- as they guide our nation in the pursuit tions of the U.S. reform movement of excellence. more closely than does U.S. and Ger- man teaching. Is this an important factor in understanding why Japan also scores among the top nations of the world in mathematics? Under- taking similar videotape observational studies of other high-performing nations and further analysis of the TIMSS teacher questionnaire data could provide insight into this question.

TIMSS’ LONG TERM UTILITY TO

72 W O R K S C I T E D

1Elley, W.B. How in the world do students read? International Association for the Evalu- ation of Educational Achievement, The Hague, Netherlands, July 1992.

2Pelgrum, H. and Plomp, T. International IEA Computers in Education Study: Pergamon Press, New York, NY, 1993.

3U.S. Department of Education. National Center for Education Statistics. Reading Literacy in the United States: Findings from the IEA Reading Literacy Study, by Binkley, M. and Williams, T. Washington, D.C.: U.S. Government Printing Office, 1996.

4U.S. Department of Education. National Center for Education Statistics. Educa- tion in states and nations: Indicators comparing U.S. states with other industrialized countries in 1991, by Phelps, R.P.; Smith, T.M.; and Alsalam, N. Washington, D.C.: U.S. Government Printing Office, 1996.

5U.S. Department of Education. National Center for Education Statistics. Interna- tional mathematics and science assessments: What have we learned? (NCES 92-011), by Medrich, E.A. and Griffith, J.E.. Washington, D.C.: U.S. Government Printing Office, 1992.

73 APPENDIX 1 ADDITIONAL TIMSS REPORTS

For more information visit the TIMSS Characterizing Pedagogical Flow: An Inves- website at: http://www.ed.gov/NCES/ tigation of Mathematics and Science Teach- timss. ing in Six Countries. Schmidt et al. Kluwer, Hingham, MA. (1996).

REPORTS PUBLISHED ON OR Mathematics Textbooks: A Comparative BEFORE NOVEMBER 20, 1996 Study of Grade Eight Texts. Howson, A.G. Pacific Educational Press, University of Pursuing Excellence. National Center for British Colombia, Vancouver, Canada. Education Statistics. U.S. Department (1995). of Education. Washington D.C. 1996. Available for downloading from the Curriculum Frameworks for Mathematics and TIMSS website at: http://www.ed.gov/ Science. Robitaille, D. F. (ed). Pacific NCES/timss. Available for sale at the Educational Press, University of British Government Printing Office. Phone Colombia, Vancouver, Canada. (1993). (202) 512-1800.

Mathematics Achievement in the Middle FORTHCOMING REPORTS School Years: IEA’s Third International Mathematics and Science Study (TIMSS). Many Visions, Many Aims: A Cross- Beaton, A.E. et al. Chestnut Hill, Bos- National Investigation of Curricular In- ton College (1996). For information tentions in School Mathematics. contact the TIMSS International Study Center, Campion Hall, Rm. 323, Bos- Many Visions, Many Aims: A Cross- ton College, Chestnut Hill, MA 02167. National Investigation of Curricular In- tentions in School Science. Science Achievement in the Middle School Years: IEA’s Third International Mathemat- A Splintered Vision: An Investigation ics and Science Study (TIMSS). Beaton, of U.S. Science and Mathematics Edu- A.E. et al. Chestnut Hill, Boston Col- cation. lege (1996). For information contact the TIMSS International Study Center, U.S. TIMSS: Mathematics and Science Campion Hall, Rm. 323, Boston College, in the Eighth Grade. Chestnut Hill, MA 02167. U.S. TIMSS: Compendium of Statistics; Third International Mathematics and Sci- 7th and 8th Grades. ence Study: Quality Assurance in Data Col- lection. Martin, M.O, and Mullis, I.V.S. U.S. TIMSS: Technical Report. Chestnut Hill, MA: Boston College (1996). The TIMSS Videotape Classroom Study: Methods and Preliminary Find- Third International Mathematics and Sci- ings. Stigler, James et al. ence Study Technical Report, Volume I: De- sign and Development. Martin, M.O., and The Education System in Germany: Kelly, D.L. Chestnut Hill, MA: Boston Case Study Findings. College (1996). The Education System in the U.S.: Case

74 Study Findings. Science in 41 Nations and 43 States.

The Education System in Japan: Case Various International and U.S. Reports Study Findings. Based on 4th Grade TIMSS Data.

Case Study Literature Review of Educa- Various International and U.S. Reports tion Policy Topics in Germany, Japan, Based on 12th Grade TIMSS Data. and the United States.

TIMSS - NAEP Link for Eighth-Grade Mathematics in 41 Nations and 43 States.

TIMSS - NAEP Link for Eighth-Grade

75 APPENDIX 2 ADVISORS TO THE U.S. TIMSS STUDY

U.S. TIMSS STEERING COMMITTEE Paul Sally The University of Chicago William Schmidt - Chair U.S. TIMSS National Research Coordi- Richard Shavelson nator, Michigan State University College Stanford University of Education Bruce Spencer Gordon Ambach Northwestern University Council of Chief State School Officers Elizabeth Stage Deborah Ball University of California University of Michigan James Taylor Audrey Champagne Global M SUNY University at Albany Kenneth Travers David Cohen University of Illinois University of Michigan Paul Williams John Dossey University of Wisconsin Illinois State University John Dossey Emerson Elliott Illinois State University National Council for Accreditation of Teacher Education VIDEOTAPE AND CASE STUDY Sheldon Glashow CONSULTANT ROUNDTABLE Harvard University MEMBERS

Larry Hedges Robert LeVine - Chair University of Chicago Harvard University

Henry Heikkinen Philip Altbach University of Northern Colorado Boston College

Jeremy Kilpatrick Jim Hiebert University of Georgia University of Delaware

Mary Lindquist Eugenia Kemble Columbus College American Federation of Teachers

Marcia Linn Dan Levine University of California at Berkeley Westat

Robert Linn Mary Lindquist University of Colorado Columbus College

76 Francisco Ramirez ADVISORS TO DEVELOPMENT Stanford University OF THIS REPORT

Paul Salley, Joe Crosswhite University of Chicago Ohio State University

Rainer Silbereisen Henry Heikkinen University of Jena, Germany University of Northern Colorado

Floraline Stevens Ross Brewer Floraline I. Stevens and Associates Exemplars

David Stevenson Eugenia Kemble U.S. Department of Education American Federation of Teachers

77 APPENDIX 3

NATIONAL AVERAGE SCORES AND STANDARD ERRORS

The 95 percent “plus or minus” confidence interval around each nation’s score is two times the standard error.

MATHEMATICS SCIENCE COUNTRY AVERAGE STANDARD AVERAGE STANDARD ERROR ERROR (AUSTRALIA) 530 4.0 545 3.9 (AUSTRIA) 539 3.0 558 3.7 BELGIUM-FLEMISH O 565 5.7 550 4.2 (BELGIUM-FRENCH) 526 3.4 471 2.8 (BULGARIA) 540 6.3 565 5.3 CANADA 527 2.4 531 2.6 (COLOMBIA) 385 3.4 411 4.1 CYPRUS 474 1.9 463 1.9 CZECH REPUBLIC 564 4.9 574 4.3 (DENMARK) 502 2.8 478 3.1 ENGLAND *O 506 2.6 552 3.3 FRANCE 538 2.9 498 2.5 (GERMANY) *O 509 4.5 531 4.8 (GREECE) 484 3.1 497 2.2 HONG KONG 588 6.5 522 4.7 HUNGARY 537 3.2 554 2.8 ICELAND 487 4.5 494 4.0 IRAN, ISLAMIC REPUBLIC 428 2.2 470 2.4 IRELAND 527 5.1 538 4.5 (ISRAEL) * 522 6.2 524 5.7 JAPAN 605 1.9 571 1.6 KOREA 607 2.4 565 1.9 (KUWAIT) 392 2.5 430 3.7 LATVIA (LSS) O 493 3.1 485 2.7 LITHUANIA * 477 3.5 476 3.4 (NETHERLANDS) 541 6.7 560 5.0 NEW ZEALAND 508 4.5 525 4.4 NORWAY 503 2.2 527 1.9 PORTUGAL 454 2.5 480 2.3 (ROMANIA) 482 4.0 486 4.7 RUSSIAN FEDERATION 535 5.3 538 4.0 (SCOTLAND) 498 5.5 517 5.1 SINGAPORE 643 4.9 607 5.5 SLOVAK REPUBLIC 547 3.3 544 3.2 (SLOVENIA) 541 3.1 560 2.5 (SOUTH AFRICA) 354 4.4 326 6.6 SPAIN 487 2.0 517 1.7 SWEDEN 519 3.0 535 3.0 SWITZERLAND O 545 2.8 522 2.5 (THAILAND) 522 5.7 525 3.7 UNITED STATES O 500 4.6 534 4.7 INTERNATIONAL AVERAGE 513 516

SOURCE: Beaton et al. (1996) Mathematics achievement in the middle school years. Table 1.1. Boston College: Chestnut Hill, MA., Beaton et al. (1996) Science achievement in the middle school years. Table 1.1. Boston College: Chestnut Hill, MA.

78 APPENDIX 4 SUMMARY OF NATIONAL DEVIATIONS FROM INTERNATIONAL STUDY GUIDELINES

Twenty-two of the 41 TIMSS countries percent of schools and students com- experienced a more or less serious de- bined. Participation rate was 63 percent viation from international guidelines for after replacements for refusals were sub- execution of the study. In 16 countries, stituted. the TIMSS International Study Center considered the deviations to be suffi- *Colombia - The pair of grades tested ciently serious to raise questions about was one grade higher than the interna- the confidence to be placed in their tional target. Average age of students in scores. These 16 nations with major dif- the upper grade was 15.7. ficulties are noted with an asterisk in this appendix, and with parentheses in Fig- *Denmark - International guidelines re- ures 1, 2, 3, 5, and 6 in this report. quiring random selection of the classrooms to receive the assessment were not *Australia - Participation rate did not followed. meet the international criterion of 75 percent of schools and students com- England - More than the international bined. Participation rate was 70 percent criterion of ten percent of schools and after replacements for refusals were sub- students were excused from the test for stituted. various reasons with resulting coverage of 89 percent of the desired population. *Austria - Participation rate did not meet Participation rate of 75 percent of the international criterion of at least 50 schools and students combined was percent participation by schools before achieved only after replacements for re- replacement. The initial participation fusals were substituted. rate was 41 percent before replacement. Participation rate was 80 percent after *Germany - The pair of grades tested replacements for refusals were substi- was one grade higher than the interna- tuted. tional target. Average student age of students in the upper grade was 14.8. One *Belgium (Flemish) - Participation rate of sixteen regions (Baden-Wuerttemberg) of 75 percent of schools and students did not participate in the study, with re- combined was achieved only after resulting coverage of 88 percent of the de- placements for refusals were substituted. sired population. Participation rate of 75 percent of schools and students com- Belgium (French) - Participation rate did bined was achieved only after replace- not meet the international criterion of ments for refusals were substituted. 75 percent of schools and students combined. Participation rate was 72 percent *Greece - International guidelines requir- after replacements for refusals were sub- ing random selection of the classrooms stituted. to receive the assessment were not followed. *Bulgaria - Participation rate did not meet the international criterion of 75

79 *Israel - Test administered only in the *Romania - The pair of grades tested was Hebrew-speaking public school system, one grade higher than the international with resulting coverage of 74 percent of target. Average student age in the up- the desired population. International per grade was 14.6. guidelines requiring random selection of the classrooms to receive the assessment *Scotland - Participation rate did not were not followed. Participation rate did meet the international criterion of 75 not meet the international criterion of percent of schools and students com- at least 50 percent participation by bined. Participation rate was 73 percent schools in the sample before replace- after replacements for refusals were sub- ment. The participation rate before re- stituted. placement was 45 percent. *Slovenia - The pair of grades tested was *Kuwait - In contrast to other nations, one grade higher than the international which tested two adjacent grades, Ku- target. Average student age was 14.8. wait tested only one grade; the ninth grade. This grade was higher than ei- *South Africa - International guidelines ther of the grades which should have requiring random selection of the class- been the international target. Average rooms to receive the assessment were not student age was 15.3. followed. Participation rate did not meet the international criterion of 75 percent Latvia (LSS) - Test administered only in of schools and students combined. Par- Latvian-speaking schools, with resulting ticipation rate was 62 percent after re- coverage of 51 percent of the desired placements for refusals were substituted. population. Because coverage falls below the international 65 percent popu- Switzerland - Test administered in 22 of lation-coverage criterion, Latvia is des- 26 cantons, with resulting coverage of ignated (LSS) for Latvian Speaking 86 percent of the desired population. Schools. *Thailand - International guidelines re- Lithuania - Test administered only in quiring random selection of the class- Lithuanian-speaking schools, with re- rooms to receive the assessment were not sulting coverage of 84 percent of the de- followed. sired population. United States - Participation rate of 75 *Netherlands - Participation rate did not percent of schools and students com- meet the international criterion of at bined was achieved only after replace- least 50 percent participation by schools ments for refusals were substituted. before replacement. The initial participation rate before replacement was 24 percent.