Civic Education and Charter Schools: Current Knowledge and Future Research Issues

Summary of Findings

Preparing young people to be good citizens has been a mission of public education since the early days of the nation. In recent years, as schools have shifted more attention to English language arts and mathematics, several groups have made a plea for renewed attention to civic education for all students. One such group is the Spencer Foundation, which promotes research to improve students’ civics knowledge and skills and their dispositions for responsible citizenship.

Of particular interest to the Spencer Foundation and others are three questions relating to the status of civic education in the rapidly growing sector of charter schools—publicly funded schools that are typically governed by an organization under a contract or charter with a state or other jurisdiction:

• How does the civics achievement of charter school students compare with that of students in traditional public schools?

• Do charter schools typically offer different types of civics instruction than traditional schools?

• How can the current body of research about civic education and charter schools be strengthened?

Center on Education Policy Graduate School of Education and Human Development The George Washington University 2

At the request of the Spencer Foundation, the Center on Education Policy (CEP) at The George Washington University addressed these questions by analyzing data from the 2010 civics assessment of the National Assessment of Educational Progress (NAEP). This is the most recent NAEP civics assessment and the only one that breaks out data for charter schools and traditional public schools. We also reviewed the broader research literature on charter schools and civic education and engagement. Our main findings are summarized below.

Statistically Significant Findings from NAEP

NAEP is the only nationally representative dataset on civics achievement and instruction in charter schools, but it has serious limitations as regards charter schools. Results from the civics assessment are available only for the nation as a whole and are not broken out by urban, suburban, or rural location. Thus, while comparisons can be made between a representative sample of all charter schools and all traditional public schools in the nation, it is not possible to compare urban charter schools with urban traditional schools, even though most charters are concentrated in urban areas and serve the diverse populations found in urban school districts.

In addition, the sample of charter school students who took the 2010 assessment was small, and the standard error rates for charter school results are high, as explained in the NAEP section of this report. Consequently, differences between charter and traditional schools that appear to be large are often not statistically significant.

We addressed this last limitation by grouping the results of the NAEP analysis into two categories: “statically significant” findings that meet NAEP’s criteria for significance, and “suggestive” differences between charter and traditional public schools that are not statistically significant but are large enough to suggest a need for further research using other methods.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 3

Our analysis of the NAEP 2010 civics data revealed a number of statistically significant differences between charter and traditional public schools, as well as several noteworthy similarities:

• Overall, civics achievement was similar for students in charter and traditional schools. At most grades, and for most racial/ethnic student groups, the civics achievement of charter school students did not differ significantly from that of traditional public school students in 2010.

• Significant differences between charter and traditional public schools were evident for a few student groups at particular grades. Hispanic 8th graders in charter schools scored significantly higher in civics than those in traditional schools. Male 12th graders in traditional public schools scored significantly higher than those in charter schools.

• There were no significant differences between charter and traditional public schools in how often students reported studying social studies. At grade 8, a large majority of both groups reported having social studies lessons every day.

• Fourth-grade teachers in both charter and traditional public schools appeared to place about the same amount of emphasis on basic civics topics. For example, similar percentages of 4th grade teachers in both school sectors reported giving “moderate” emphasis in their social studies classes to politics and government, foundations of U.S. democracy, the role of citizens in U.S. democracy, and other topics.

• Some differences were evident in the civics topics studied by students in charter and traditional public schools. At grade 8, for example, significantly higher percentages of charter school students than of traditional school students reported studying the U.S. Congress, how laws are made, the court system, and the U.S. Constitution. At grade 12, the percentages of students who did not study international organizations and other countries’ governments were higher in

Center on Education Policy Graduate School of Education and Human Development The George Washington University 4

traditional schools than in charter schools. Roughly equal percentages of 12th graders in charter and traditional public schools reported taking an Advanced Placement government and politics course.

• Some significant differences emerged in how social studies was taught in charter and traditional public schools. At grade 8, significantly higher percentages of charter school students than of traditional school students reported taking part in role play, mock trials, and dramas one to two times per month. A greater share of charter school 8th graders also reported responding to short-answer questions in their social studies classes almost every day. At grade 12, charter school students wrote long answers to questions more often than their peers in traditional schools.

• Similar proportions of 4th grade teachers in charter and traditional public schools reported basing their social studies programs on state curriculum standards. Roughly equal percentages of 4th grade teachers in both school sectors also reported that their school’s social studies programs were structured around district and national assessment results. However, significantly higher percentages of traditional school teachers said their social studies programs were based to a large extent on district curriculum standards, district recommendations, and national curriculum standards.

• In general, students in charter and traditional public schools had similar responses about the availability of resources at home that could contribute to civics knowledge. Roughly equal percentages of charter and traditional public school students reported having computers and newspapers at home, and the two groups were similar in how often they discussed what they had studied in school with someone at home.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 5

Suggestive Results from NAEP

Although not statistically significant, the following differences in NAEP results between charter and traditional public schools were large enough to warrant further investigation, perhaps using other data sources or research methods:

• Some suggestive differences were found in civics achievement. At grade 12, the average civics scores were higher for traditional public school students than for charter school students, though not significantly so. This was the case both for students overall and for some student subgroups. At grade 8, while the average scores for students overall were not widely different in the two sectors, there was a consistent pattern of apparently higher scores for charter school students in certain subgroups, such as African American and low-income students.

• NAEP data suggest some possible differences between charter and traditional public schools in the sources of their 4th grade social studies curricula. A noticeably higher percentage of charter school teachers reported structuring their social studies program “to a large extent” around school-based standards, although the differences were not statistically significant. In addition, higher shares of traditional public school teachers reported structuring their social studies programs to some extent on teacher discretion—a counterintuitive result that warrants further exploration.

Findings from Our Review of Other Research

In addition to analyzing NAEP data, we reviewed other research literature on civic education and charter schools, as well as more general studies of civic education in all public and private schools. This review produced one major finding:

• Very little empirical research exists on civic education in charter schools—or, for that matter, in public schools in general. Only a small number of published

Center on Education Policy Graduate School of Education and Human Development The George Washington University 6

dissertations and papers have specifically examined civic education in charter schools. A limited number of studies have looked at best practices for teaching civics and promoting civic education in public schools, but these studies usually focused on particular districts, schools, or programs. In general, the research base is inadequate to assess the quality of civic education in any type of school or to inform decisions about this field.

Recommendations for Further Research

As the analyses described in this report make clear, many key questions about civics achievement, civic education, and civic engagement in charter schools cannot be answered because research is scarce and existing data have serious limitations. In fact, this is true for public schools in general, not just charter schools.

CEP recommends several steps for strengthening the current body of research and practical knowledge about civic education in all types of public schools, whether charter or traditional.

Recommendations Related to NAEP

We recommend that NAEP administrators take the following steps to improve NAEP data on civic education:

1. “Oversample” charter schools for the next administration of the NAEP civics assessment to include a larger representative sample of these schools. This would lead to lower standard error rates in the charter school data and would allow researchers to reach firmer conclusions about civics achievement and instruction in charter schools compared with traditional public schools.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 7

2. Include data on civics achievement in NAEP’s Trial Urban District Assessment (TUDA), which produces district-level data on student achievement and other variables for participating urban districts. Currently, the TUDA program is limited to mathematics, reading, writing, and science.

3. Include data for charter schools in participating TUDA districts, since many charter schools are located in urban areas. This recommendation would produce data that researchers could use to compare results for charter and traditional public schools in the same cities, even if the charter results are not included in other district-level results reported by NAEP. 1

Recommendations for General Research on Civic Education in All Public Schools

In addition to strengthening NAEP data, we recommend several steps to improve knowledge and practice in civic education in all public schools:

4. Hold a workshop of researchers and leaders with expert knowledge about civic education to frame a research agenda in this area.

5. Encourage additional research to determine whether apparent differences in NAEP results between charter and traditional schools are indeed real and what factors might explain these disparities. Issues for further study could include both the statistically significant and “suggestive” differences found in NAEP, such as the apparent differences between the two school sectors in 12th grade civics achievement, the achievement of certain student subgroups, and instructional methods.

1 Due to a policy change in 2009, NAEP results for districts in the TUDA no longer include charter schools that are located within the geographic boundaries of an urban district but are operated independently of the district and are not included in the district’s report of adequate yearly progress (AYP) under the No Child Left Behind Act. School districts vary in terms of whether the charter schools within their boundaries are included in district AYP results.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 8

6. Encourage more detailed and extensive research on student achievement in civics in charter and traditional public schools. This research should include studies in a limited number of states with civics-related standards and assessments and should compare students in charter and traditional public schools in similar districts and locations (urban, suburban, and rural).

7. Once the research in the preceding recommendation has been completed, conduct follow-up studies to determine why one group of students is outperforming the other. Possible variables to be examined include differences in:

• Teacher credentials or experience

• Curriculum and instructional practices or other school-level inputs such as time spent on civics-related subjects

• School resources

• Demographic characteristics such as socioeconomic status, parents’ education, level of engagement, and civic participation in the home

8. Promote more in-depth studies at the state and district levels of how civics is taught in charter and traditional public schools and which instructional methods are most effective. This could include case studies of civic education in selected school districts, particularly urban areas with concentrations of charter schools. These studies might investigate whether differences exist between charter schools and traditional public schools in such areas as the following:

• Using “active” methods of teaching civics (debates, mock trials, etc.)

• Encouraging students to express opinions in class about civics topics

• Encouraging civic engagement among students

9. Support more thorough empirical research on best practices for improving civic engagement for students in traditional public, charter, or private schools. This might include in-depth studies of whether particular characteristics of private schools, especially Catholic schools, encourage civic engagement and are replicable in public schools.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 9

The sections of this report that follow provide background information on charter schools and civic education, describe the detailed findings from our analysis of NAEP data in charter and traditional public schools, and summarize findings from our review of other research literature.

Background

Civic education has been defined in various ways, but by most definitions, it encompasses the study of government and its workings and of the rights and responsibilities of citizenship (Collins English Dictionary, 2012). The research literature on this subject makes a further distinction between two aspects of civic education: civics knowledge, which “refers to facts about U.S. history and government that can be readily measured with close- ended surveys and tests”; and civic engagement, which encompasses “participation in civic institutions through formal means, such as voting, and informal means, such as following current events and discussing politics with friends and family” (Rubin, 2007, p. 450). Other indicators of civic engagement often include community service and volunteerism; political participation and advocacy; community engagement; and political discussion (Feinberg & Doppen, 2010). In short, civic education is a vital contributor to the development of responsible, democratic citizens.

What Is the Status of Civic Education?

In recent years, researchers and advocacy groups have pointed to evidence suggesting that schools are giving insufficient attention to civic education. With the emphasis on raising student achievement in English language arts and mathematics to meet the accountability requirements of the No Child Left Behind Act, many school systems have reduced instructional time for other subjects, such as social studies and civics (Center on Education Policy, 2008). Similarly, a study published by the Center for Information and Research on

Center on Education Policy Graduate School of Education and Human Development The George Washington University 10

Civic Learning and Engagement (CIRCLE) indicated that educational resources are being shifted away from social studies towards subjects tested for accountability, such as English language arts and math (Godsay, Henderson, Levine, & Littenberg-Tobias, 2012). This same study also found that 21 states require social studies tests for K-12 students, and only 9 states require students to pass a social studies exam to graduate from high school.

A related issue is whether students are learning enough civics to become well-informed, engaged citizens. The NAEP data on civics achievement, summarized in the next section, show that achievement in civics increased between 1998 and 2010 at grade 4, stayed about the same at grade 8, and rose then fell back to its 1998 level at grade 12. While a majority of students performed at the NAEP “basic” level of achievement in civics in 2010, only about one-quarter or fewer met NAEP’s definition of “proficient” performance (which tends to be more rigorous than many states’ definition of proficiency, as noted in the NAEP section of this report.) Other studies described in the literature review section have concluded that an unacceptably large share of students lack knowledge of fundamental aspects of U.S. government and are ill-prepared to exercise their civic responsibilities.

Why Look at Civic Education in Charter Schools?

Charter schools are a rapidly growing sector of the public education system. From 1999– 2000 to 2009–10, the number of students enrolled in public charter schools more than quadrupled from 0.3 million to 1.6 million students. In 2009–10, about 5% of all public schools were charter schools (U.S. Department of Education, 2012a).

Charter schools tend to be far more concentrated in urban areas than are public schools in general, as shown in figure 1. In 2009-10, a majority (55%) of charter schools were located in cities, 21% in suburban areas, 8% in towns, and 16% in rural areas (U.S. Department of Education, 2012a). By contrast, the percentage of all public schools located in cities (25%) was much smaller, while the percentage in rural areas (33%) was much larger (U.S. Department of Education, 2012b).

Center on Education Policy Graduate School of Education and Human Development The George Washington University 11

Figure 1. Location of charter schools and all public schools, 2010

Charter schools All public schools

16% 25% 33% 8%

55%

21%

14% 28%

City Suburban Town Rural City Suburban Town Rural

Figure reads: In school year 2009-10, 55% of the nation’s charter schools were located in cities. Source: U.S. Department of Education, 2012a; 2012b.

In addition, charter schools as a group enroll higher percentages of racial/ethnic minority students than public schools in general, as shown in figure 2. Altogether, about 62% of charter school students were racial/ethnic minority students in 2010, compared with about 46% of all public school students. Since 1999-2000, the percentages of Hispanic and Asian American students in charter schools have increased, while the percentages of white and African American students have decreased (U.S. Department of Education, 2012a).

Center on Education Policy Graduate School of Education and Human Development The George Washington University 12

Figure 2. Percentage of students by race/ethnicity in charter schools and all public schools, 2010

Charter schools All public schools

1% 1% 1% 3% 4% 4%

26% 37% 23%

54%

15% 30% White White African American African American Hispanic Hispanic Asian American Asian American Alaska Native/Pacific Islander Alaska Native/Pacific Islander Other Two or more races

Figure reads: In school year 2009-10, 37% of students enrolled in public charter schools were white. Note: Percentages may not total 100% due to rounding. Source: U.S. Department of Education, 2012a; 2012c.

One-third (33%) of charter schools were high-poverty schools in 2009-10, meaning that at least three-fourths of their students were eligible for free or reduced-price school lunch (U.S. Department of Education, 2012a). Just 20% of all public schools were high-poverty schools (U.S. Department of Education, 2012c).

To sum up, the majority of schools in the rapidly growing charter sector are concentrated in cities. Consistent with this location, charter schools enroll higher percentages of minority and low-income students—groups with lower than average civics achievement

Center on Education Policy Graduate School of Education and Human Development The George Washington University 13

and fewer opportunities for civic engagement, according to research described later in this report.

Some charter school advocates maintain that the freedom from regulation and the flexible organization, curriculum, and teaching found in charter schools can have a positive impact on students’ civic education and engagement, as noted in the literature review section of this report. But very few empirical studies have examined the link between civic education and the learning environment in charter schools—or in traditional public schools.

If all public school students are to be well-prepared for citizenship, it is important to know more about civics achievement and civic education in all types of public schools.

Findings from NAEP

In 1998, 2006, and 2010, the National Assessment of Educational Progress (NAEP) tested the civics knowledge of a nationally representative sample of students at grades 4, 8, and 12. Data from the most recent assessment in 2010 were disaggregated for the first time according to whether students attended a charter school or a traditional public school, making it possible to compare the two groups.

In this section, we explore differences and similarities between charter and traditional public schools in their students’ performance on the 2010 NAEP civics assessment and their approaches to civics instruction. Before delving into these comparisons, we first provide background on the NAEP civics assessment and its limitations, describe our methodology, and briefly summarize national trends in civics achievement for students overall. (Studies of civics achievement other than NAEP are described in a separate literature review section of this report.)

Center on Education Policy Graduate School of Education and Human Development The George Washington University 14

Background on the NAEP Civics Assessment and Its Limitations

As the only civics assessment in the U.S. based on a nationally representative sample of students, NAEP is still the best source of data available for exploring civics achievement and instruction at the national level. The 2010 NAEP civics assessment measured the knowledge of students in grades 4, 8, and 12 in the following civics content areas (U.S. Department of Education, 2011a):

• What are civic life, politics, and government?

• What are the foundations of the American political system?

• How does the government established by the Constitution embody the purpose, values, and principles of American democracy?

• What is the relationship of the United States to other nations and to world affairs?

• What are the roles of citizens in American democracy?

In addition to assessing achievement in civics and other subjects, NAEP collects information from participating students, teachers, and schools on hundreds of “background variables,” as they are commonly known, that relate to student achievement. Three broad types of background data are available:

• Student demographic categories (such as gender, race/ethnicity, eligibility for free/reduced price lunch, English language learner status)

• Contextual and policy information (such as type of school, absenteeism, teacher qualifications, availability of computers and other resources)

• Subject-specific information (such as student attitudes toward the subject, time spent on the subject per week, instructional methods, emphasis on specific topics)

Center on Education Policy Graduate School of Education and Human Development The George Washington University 15

Data from these questions about background variables form the basis of many of the findings from NAEP described below.

While the NAEP civics dataset is important, it has some serious limitations. First, the civics results are reported only at the national level. By contrast, results are available in reading and math, and less frequently in science and writing, for individual states and for large urban districts participating in the NAEP TUDA program. Second, the civics data cannot be disaggregated by school location (urban, suburban, rural). Therefore, we cannot compare differences between charter and traditional schools in urban areas, where most charter schools are located. Third, while NAEP has reported overall trends in civics achievement since 1998, disaggregated data for charter schools and traditional public schools did not become available until 2010, so it is not possible to track progress over time for both types of schools.

The largest problem with the NAEP dataset for charter schools, however, is sample size. NAEP is administered to a sample of public school students nationally, and the subset of tested students in charter schools is small. Consequently, the standard error of measurement—which indicates how accurately a particular sample of test-takers represents the entire test-taking population—can be very large for the charter school results. The greater the standard error is, the more the average performance of a sample may deviate from the actual average performance of the overall population, which diminishes the significance of the findings. These large standard errors explain why many of the results described below, which appear to indicate large differences between charter and traditional public schools, are not statistically significant. As explained in the methodology discussion, the statistically significant findings are handled differently in this report from differences that appear to be sizable but are not statistically significant.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 16

Methodology for the NAEP Analysis

CEP consultants used NAEP’s online data explorer tool2 to analyze achievement data from the NAEP 2010 civics assessment for charter schools and traditional public schools. (Although the disaggregated data for charter and traditional schools were available online, they had not been included in NAEP’s own 2010 civics report.) We also analyzed data from the 2010 assessment on the background variables that were most closely related to civics curriculum and instruction—specifically, whether students participate in debates, discuss current events, or learn how to write opinion letters—along with some student characteristics. We did not examine background variables related to school administration, such as school enrollment or grade levels taught at the school.

A fair number of background variables had insufficient data (too few responses to meet NAEP reporting requirements), and many background questions asked at one grade level were not asked at another. Therefore, few of our analyses could be conducted across the 4th, 8th, and 12th grade levels.

In this report, findings that are statistically significant are referred to as “significant” and presented first. Differences between charter and traditional public schools that are large but not statistically significant are referred to as “suggestive” and presented after the significant findings. The term “suggestive” is intended to point to possible differences between charter and traditional schools that may warrant further research using other methods.

Trends in Civics Achievement for Students Overall

Trends in overall student achievement since 1998 on the NAEP civics assessment vary by grade level, as detailed in the NAEP 2010 civics report (U.S. Department of Education,

2 http://nces.ed.gov/nationsreportcard/naepdata/dataset.aspx.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 17

2011b). Compared with the 1998 and 2006 NAEP civics assessments, the average score in 2010 showed the following trends (displayed in figure 3):

Grade 4. Higher than the scores in both years

• Grade 8. Not significantly different from the score in either year

• Grade 12. Lower than the score in 2006 but not significantly different from the

• score in 1998

Figure 3. Trends in average NAEP civics scores at three grades

Grade 4 Grade 8 Grade 12 300 300 300 250 250 250 200 150* 154* 157 200 150 150 151 200 150 151* 148 150 150 150 100 100 100 50 50 50 0 0 0 1998 2006 2010 1998 2006 2010 1998 2006 2010

*Significantly different from 2010. Source: National Assessment of Educational Progress, Civics 2010, p. 1.

Sizable gaps persist in the civics achievement of students from different racial/ethnic groups. In 2010, white students scored 24 points higher on average than African American students and 27 points higher than Hispanic students (on a scale of 0-300). Neither the black-white gap nor the Hispanic-white gap changed significantly from 2006 to 2010, but both gaps narrowed compared with 1998.

NAEP also reports results in terms of the percentages of students reaching three performance levels—basic, proficient, and advanced. The NAEP basic level “denotes partial mastery of the knowledge and skills fundamental for proficient work at each grade”; the proficient level “represents solid academic performance to which all students at each grade

Center on Education Policy Graduate School of Education and Human Development The George Washington University 18

assessed should aspire”; and the advanced level “represents superior performance” (U.S. Department of Education, 2011b). Some analyses have found the NAEP basic level to be closer to the definitions of “proficient” used in most states for their own state tests, while the NAEP “proficient” level is more aspirational, signaling where students should be (CEP, 2010). Box A gives examples of the types of skills in civics demonstrated by students who perform at each of the three NAEP achievement levels.

Box A. Examples of civics skills demonstrated by students at NAEP achievement levels

Basic • Recognize taxes as the main source of government funding (grade 4). • Identify a right protected by the First Amendment (grade 8). • Interpret a political cartoon (grade 12).

Proficient • Identify a purpose of the U.S. Constitution (grade 4). • Recognize a role performed by the Supreme Court (grade 8). • Define the term “melting pot” and argue if it applies to the U.S. (grade 12).

Advanced Explain two ways countries can deal with shared problems (grade 4). Name two actions citizens can take to encourage Congress to pass a law (grade 8). • Compare the citizenship requirements of the U.S. to other countries (grade 12). • •Source: National Assessment of Educational Progress, Civics 2010, p. 2.

Although a majority of students at all grades reached at least the basic level of achievement on the 2010 assessment, only about one-fourth or fewer scored at or above the proficient level. And while the percentages of students reaching the basic and proficient levels have increased at grade 4 since 2006, they have not changed significantly at grades 8 and 12. Nor have there been significant changes in the percentages of students scoring at the advanced level at any of three grade levels. The 2010 results by achievement level are as follows:

Basic level. Nationally, 77% of 4th graders, 72% of 8th graders, and 64% of 12th

• graders performed at or above the basic level in civics in 2010. The 2010

Center on Education Policy Graduate School of Education and Human Development The George Washington University 19

percentages of students scoring at or above basic were higher than those in 2006 and 1998 at grade 4, but not significantly different from previous assessment years at grades 8 and 12.

Proficient level. Nationally, 27% of 4th graders, 22% of 8th graders, and 24% of 12th

• graders performed at or above proficient in 2010. The 2010 percentages of students reaching the proficient level were higher than those in 2006 and 1998 at grade 4, not significantly different from the percentages in the previous assessment years at grade 8, and lower than 2006 at grade 12.

Advanced level. Just 2% of 4th graders, 1% of 8th graders, and 4% of 12th graders

• performed at the advanced level in civics in 2010. The advanced percentages have not changed significantly compared with previous assessment years at any of the three grades.

Comparison of Charter and Traditional Public Schools

A central question examined in this report is whether students in charter schools perform better in civics than those in traditional public schools. Table 1 shows the average scores for these two groups at grades 4, 8, and 12. Several points are apparent from this analysis:

• At most grade levels and for most racial/ethnic and demographic groups, the civics achievement of charter school students did not differ significantly from that of traditional public school students.

• For a few student groups at particular grade levels, significant differences did emerge. Hispanic 8th graders scored significantly higher in charter schools than in traditional public schools. Male 12th graders scored significantly higher in traditional public schools than in charter schools.

• There were some suggestive, though not statistically significant, differences in achievement that deserve further study. At grade 12, average scores were higher in

Center on Education Policy Graduate School of Education and Human Development The George Washington University 20

traditional public schools than in charter schools—both for students overall (the “all students” row in table 1) and for every subgroup with sufficient data. It is possible that more low-achieving students drop out of traditional schools before 12th grade, leaving a group of 12th graders in traditional schools that is higher-performing on average than the comparable group in charter schools. But this is speculation, and a different pattern might emerge if data were available to compare charter and traditional students in urban areas.

• At grade 8, average scores were consistently higher for charter school students than for traditional school students—both for students overall and for every subgroup with sufficient data. These differences were not statistically significant, except for the Hispanic subgroup as already noted, but were large enough for African American and low-income students to suggest a need for further study.

• Although the average grade 12 civics scores appear to be lower than the averages in grades 4 and 8, NAEP develops its scoring scales independently for each grade, so scores across grades are not comparable.

Table 1. Average NAEP scores in civics for charter and traditional public school students, 2010

Grade 4 Grade 8 Grade 12 Student group Charter Traditional Charter Traditional Charter Traditional All students 158 156 155 150 121 148 Low-income 142 143 149 135 117 132 White 173 165 ‡ 159 ‡ 155 African American 144 142 146 133 ‡ 128 Hispanic 141 139 161* 134 137 135 Male 156 153 153 149 116 148* Female 159 159 156 151 126 147 English language ‡ 123 ‡ 104 ‡ 99 learners Students with ‡ 133 ‡ 117 ‡ 108 disabilities

Center on Education Policy Graduate School of Education and Human Development The George Washington University 21

Table reads: At grade 4, the average civics score in 2010 was 158 for students attending charter schools and 156 for students attending traditional public schools. This 2-point difference is not statistically significant at the .05 level. *The difference in average scores between charter and traditional public schools is statistically significant at the .05 level. ‡NAEP reporting standards were not met; too few cases. Note: The NAEP civics scale ranges from 0 to 300 at each grade. The scale was derived independently at each grade, so scale scores across grades cannot be compared. For example, equal scores on the grade 4 and grade 8 civics scales do not imply equal levels of civics achievement.

Time Spent on Social Studies

For students overall, the NAEP background data reveal a relationship between time spent on social studies and civics achievement. As shown in table 2, students who reported studying social studies almost every day scored significantly higher on the NAEP civics assessment than students who reported studying it less often.

Table 2. Average NAEP scores in civics by how often students reported studying social studies, 2010

Grade level Hardly ever 1-2 times/month 1-2 times/week Almost every day Grade 8 135 136 142* 155* Grade 12 140 146* 150* 156*

Table reads: Grade 8 students from all types of public schools who reported studying social studies every day scored 155, on average, on the NAEP civics assessment—significantly higher than the average scores of students who reported spending less time studying social studies. *The difference between this score and the score in the column to its left is statistically significant at the .05 level. Note: The NAEP civics scale ranges from 0 to 300 at each grade. The scale was derived independently at each grade, so scale scores across grades cannot be compared. For example, equal scores on the grade 4 and grade 8 civics scales do not imply equal levels of civics achievement.

In light of the importance of instructional time, we examined whether students in charter schools spent more time studying social studies than students in traditional public schools. The results of this analysis, displayed in table 3, reveal the following findings:

Center on Education Policy Graduate School of Education and Human Development The George Washington University 22

• There were no statistically significant differences between charter and traditional public schools in how often students reported studying social studies. At grade 8, a large majority of students in both sectors reported studying social studies almost every day. Although the percentage of “almost every day” responders appears to be somewhat higher for charters than for traditional schools, the difference is not significant.

• At grade 12, the results for charters and traditional schools were very similar. Fewer 12th graders in both school sectors reported taking “social studies” every day than did 8th graders. This is probably because high school students are often required to take more specific courses, such as U.S. or world history, economics, government, or geography, which they may not report as social studies courses.

Table 3. Percentage of students in grades 8 and 12 choosing various responses to the question, “In this school year, how often have you studied social studies?” 2010

Hardly ever 1-2 times/ month 1-2 times/week Almost every day Grade 8 Charter 4% 4% 11% 81% Traditional 7% 6% 18% 69% Grade 12 Charter 41% 11% 15% 33% Traditional 43% 9% 16% 32%

Table reads: At grade 8, 4% of charter school students reported that they hardly ever studied social studies in the school year of the NAEP assessment.

Instructional Content

We analyzed NAEP data on the extent to which 4th grade teachers in charter schools and traditional public schools emphasized various civics topics in their social studies classes. Several conclusions can be drawn from the data in table 4:

Center on Education Policy Graduate School of Education and Human Development The George Washington University 23

• Fourth-grade teachers in both types of schools seem to place about the same amount of emphasis on most of the topics listed.

• There is only one statistically significant difference between the two groups of teachers. The percentage of teachers who reported emphasizing the role of the U.S. in the world to a “small extent” was higher in traditional schools than in charter schools. It is not clear, however, that charter schools give this topic more emphasis in light of the percentage of charter school teachers who reported that they “never” emphasize this topic.

• There is suggestive evidence that 4th grade teachers in charter schools may be placing more emphasis than traditional public school teachers on the topics of economic changes, the environment, various cultures, and world affairs. But the findings are not statistically significant, and this issue would need to be studied further.

Table 4. Percentage of 4th grade teachers choosing various responses to the question, “To what extent have you emphasized the following topics in your social studies class this year?” 2010

Topic Not at all Small extent Moderate extent Large extent Change in U.S. democracy Charter 40% 35% 16% 9% Traditional 32% 42% 21% 5% Economic changes Charter 8% 27% 59% 7% Traditional 10% 44% 37% 9% Environment and society Charter 2% 26% 43% 30% Traditional 5% 35% 45% 15% Foundations of U.S. democracy Charter 25% 39% 29% 7% Traditional 19% 37% 34% 10%

Center on Education Policy Graduate School of Education and Human Development The George Washington University 24

Topic Not at all Small extent Moderate extent Large extent Various cultures Charter 3% 18% 42% 38% Traditional 4% 28% 48% 21% Politics and government Charter 10% 41% 40% 9% Traditional 12% 47% 35% 6% Role of U.S. in the world Charter 34% 26% 34% 6% Traditional 26% 46%* 23% 5% Role of citizens in U.S. democracy Charter 21% 35% 32% 13% Traditional 12% 43% 34% 10% Space and place Charter 9% 41% 28% 22% Traditional 14% 41% 36% 10% Technological changes Charter 6% 39% 41% 14% Traditional 11% 42% 40% 8% U.S. Constitution Charter 22% 47% 27% 4% Traditional 19% 41% 32% 8% World affairs Charter 18% 53% 23% 6% Traditional 31% 49% 17% 4%

Table reads: At grade 4, 40% of teachers in charter schools reported that they give no emphasize at all to change in U.S. democracy in their social studies classes. *The difference in percentages between charter and traditional public schools is statistically significant at the .05 level.

In grades 8 and 12, NAEP surveyed students about the civics-related topics studied in school. Table 5 shows the results of our analysis of these data.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 25

• At grade 8, significantly higher percentages of charter school students than of traditional public school students reported studying the following areas: the U.S. Congress, how laws are made, the court system, and the U.S. Constitution.

• At grade 12, the percentages of students who did not study international organizations and other countries’ governments were significantly higher in traditional schools than in charter schools.

• Roughly equal percentages of 12th graders in charter and traditional public schools reported taking an Advanced Placement government and politics course.

Table 5. Percentage of students responding “yes” or “no” to the question, “During this school year, have you studied any of the following topics?” 2010

Grade 8 Grade 12 Topic Yes No Yes No U.S. Congress Charter 95%* 4% 69% 26% Traditional 78% 15%* 66% 31% How laws are made Charter 87%* 9% 68% 27% Traditional 70% 21%* 61% 36% International organizations Charter 26% 44% 45% 38% Traditional 33% 43% 43% 48%* Other countries' governments Charter 38% 44% 54% 37% Traditional 41% 43% 46% 47%* Parties, elections, and voting Charter 85% 10% 73% 24% Traditional 74% 18% 68% 29% President and Cabinet Charter 65% 19% 61% 32%

Center on Education Policy Graduate School of Education and Human Development The George Washington University 26

Grade 8 Grade 12 Topic Yes No Yes No Traditional 61% 27% 58% 37% State and local government Charter 80% 12% 70% 23% Traditional 70% 19% 68% 29% Court system Charter 80% 11% 67% 31% Traditional 64% 25%* 61% 36% U.S. Constitution Charter 99% 1% 69% 26% Traditional 82% 13%* 67% 31% Taken an AP U.S. government and politics course Charter n/a n/a 16% 84% Traditional n/a n/a 16% 84%

Table reads: Among 8th grade students attending charter schools, 95% reported studying the U.S. Congress in the school year of the NAEP assessment, compared with 78% of 8th graders in traditional public schools. *The difference in percentages between charter schools and traditional public schools is statistically significant at the .05 level. Note: The yes and no responses do not necessarily total 100% because students could also respond “don’t know.”

Instructional Methods

Are there differences in how civics is taught in charter and traditional public schools? Some evidence can be found in the responses of students to questions about what they do in social studies classes. The results for grade 8, shown in table 6, reveal the following main findings:

• Eighth graders attending charter schools appeared to experience a more active approach to social studies instruction than those in traditional public schools. Significantly greater percentages of charter school students reported taking part in role playing, mock trials, and dramas 1-2 times per month. More than half of traditional public school students reported never doing these activities. In addition,

Center on Education Policy Graduate School of Education and Human Development The George Washington University 27

a significantly higher proportion of traditional school students than of charter school students said they never take part in debates or panel discussions in their social studies classes.

• Significantly greater proportions of charter school students than of traditional public school students reported that they respond to short-answer questions almost every day in their social studies classes.

• Although not statistically significant, the data show suggestive differences in a few areas. Specifically, 8th grade students in charter schools may take tests or quizzes in social studies class more frequently than those in traditional public schools, while students in traditional schools may spend more time watching TV, movies, or videos in social studies class than do charter school students.

Table 6. Percentage of grade 8 students choosing various responses to the question, “How often do you [do the following activities] in social studies?” 2010

Few times/ 1-2 times/ 1-2 times/ Almost Activity Never year month week every day Take a test or quiz Charter # 2% 21% 63% 14% Traditional 2% 3% 37% 49% 9% Take part in debates or panel discussions Charter 21% 30% 20% 18% 11% Traditional 46%* 19% 18% 11% 5% Take part in role-playing, mock trials, dramas Charter 32% 22% 28%* 14% 4% Traditional 54% 22% 15% 6% 3% Watch movies or videos Charter 3% 22% 44% 19% 11% Traditional 5% 18% 39% 26% 13% Write a report Charter 11% 31% 44% 11% 3% Traditional 16% 33% 38% 10% 3%

Center on Education Policy Graduate School of Education and Human Development The George Washington University 28

Work on a group project Charter 5% 25% 41% 22% 6% Traditional 10% 25% 43% 17% 5% Write an opinion letter Charter 61% 23% 12% 3% 2% Traditional 68% 18% 8% 4% 2% Write long answers to questions Charter 4% 14% 48% 33% Not asked Traditional 7% 20% 39% 35% Not asked Write short answers to questions Charter 5% 3% 14% 37% 40%* Traditional 7% 8% 20% 38% 27%

Table reads: At grade 8, 2% of charter school students reported that they took a test or quiz a few times per year. # Rounds to 0. *The difference in percentages between charter schools and traditional schools is statistically significant at the .05 level.

The same questions about classroom experiences in social studies were asked of grade 12 students. Several findings can be gleaned from the results, displayed in table 7:

• The percentage of grade 12 students who take part in role-playing, mock trials, or dramas “a few times a year” was significantly higher for traditional school students than for charter school students. This distinction is hard to interpret, however, because there were no significant differences for the other responses about how frequently students did these activities.

• A significantly higher share of 12th graders in charter schools than in traditional schools reported writing short answers to questions almost every day in their social studies classes. Charter school students also wrote long answers to questions more often than their traditional school counterparts, as inferred from the significantly higher percentage of traditional school students who reported never doing this in social studies.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 29

• Two other differences are apparent but difficult to interpret. Traditional school students were significantly more likely than charter school students to report taking tests or quizzes once or twice a month. At the same time, an apparently larger percentage of charter school students reported taking tests or quizzes almost every day, although the difference was not statistically significant.

Table 7. Percentage of grade 12 students choosing various responses to the question, “How often do you [do the following activities] in social studies?” 2010

Few times/ 1-2 times/ 1-2 times/ Almost Activity Never year month week every day Take test or quiz Charter 4% 2% 19% 52% 23% Traditional 3% 4% 33%* 52% 8% Take part in debates or panel discussions Charter 35% 18% 21% 13% 13% Traditional 31% 22% 23% 16% 8% Take part in role-playing, mock trials, dramas Charter 58% 15% 17% 9% 1% Traditional 55% 26%* 12% 4% 3% Watch movies or videos Charter 9% 28% 31% 11% 21% Traditional 6% 27% 38% 20% 9% Write a report Charter 13% 25% 36% 22% 4% Traditional 15% 35% 38% 10% 2% Work on a group project Charter 12% 19% 37% 21% 11% Traditional 11% 30% 38% 16% 4% Write an opinion letter Charter 65% 15% 7% 10% 4% Traditional 70% 20% 7% 3% 1% Write long answers to questions Charter 15% 24% 33% 28% Not asked Traditional 29%* 28% 26% 17% Not asked

Center on Education Policy Graduate School of Education and Human Development The George Washington University 30

Few times/ 1-2 times/ 1-2 times/ Almost Activity Never year month week every day Write short answers to questions Charter 4% 13% 18% 31% 34%* Traditional 8% 11% 24% 40% 16% Table reads: In grade 12, 4% of charter school students reported that they take a test or quiz one to two times per month. *The difference in percentages between charter schools and traditional schools is statistically significant at the .05 level.

Sources of Social Studies Curriculum

What do traditional and charter schools use as a guide to plan their social studies and civics instruction? Table 8 illustrates the extent to which social studies programs are based on national, state, or local standards; various types of assessments; teacher discretion; and other sources. We could address this question only at grade 4 because NAEP data were not available at all for grade 8 and were very limited and inconclusive at grade 12.

Several implications emerge from the analysis of these data:

• Similar proportions of 4th grade teachers in charter and traditional public schools reported that their social studies programs were based on state curriculum standards. This is understandable because both types of schools are held accountable for students learning the knowledge and skills in their state’s standards.

• Significantly higher percentages of traditional school teachers than of charter school teachers reported basing their social studies programs to a large extent on district curriculum standards, district recommendations, and national curriculum standards. This is likely because charter schools, by definition, are typically exempted from many district requirements.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 31

• Similar percentages of teachers in traditional and charter schools reported that their schools’ social studies programs were structured around district and national assessment results. The apparent differences in the percentages that structured their programs around state assessment results were not significant.

• Although the differences are not statistically significant, the data suggest that charter school teachers may base their social studies programs on their own school standards to a greater extent than public school teachers.

• There is also suggestive evidence that traditional schools structure their social studies programs according to teacher discretion to a greater extent than charter schools. This is a counterintuitive result that warrants further exploration.

Table 8. Percentage of teachers choosing various responses to the question, “To what extent is your school's social studies program for 4th graders structured according to [the following sources]” 2010

Source Not at all Small extent Moderate extent Large extent Commercial programs Charter 34% 31% 20% 15% Traditional 40% 28% 21% 11% Discretion of teachers Charter 40% 23% 13% 24% Traditional 16% 38% 26% 19% District recommendations Charter 58%* 24% 18% # Traditional 25% 21% 29% 26% District or school assessment results Charter 28% 28% 19% 24% Traditional 24% 28% 24% 25%

Center on Education Policy Graduate School of Education and Human Development The George Washington University 32

Source Not at all Small extent Moderate extent Large extent District curriculum standards Charter 12% 32% 18% 38% Traditional 6% 8% 16% 70%* National assessment results Charter 63% 30% 8% # Traditional 62% 23% 12% 4% School-based curriculum standards Charter 13% 6% 17% 65% Traditional 28% 22% 16% 34% State curriculum standards Charter # 2% 15% 83% Traditional 2% 3% 12% 83% State assessment results Charter 17% 31% 40% 12% Traditional 37% 20% 17% 27% National curriculum standards Charter 16% 43% 32% 9% Traditional 25% 23% 32% 20%*

Table reads: Thirty-four percent of charter school teachers reported that their social studies program is not at all structured according to commercial programs. # Rounds to 0 *The difference in percentages between charter schools and traditional schools is statistically significant at the .05 level.

Students’ Home Environment

NAEP collects some background information about students’ home environments that gives further clues about differences between charter and traditional school students. The results from the background questions asked of students who took the 2010 civics assessment are contained in the next three tables.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 33

Table 9, which deals with computers, magazines, and newspapers in the home, shows the following:

• Roughly equal percentages of charter and traditional public school students reported having computers and newspapers at home.

• Traditional public school students were more likely than charter school students to report having magazines; this finding is significant at grade 12.

Table 9. Percentage of students reporting they have a computer, magazines, or newspaper at home, 2010

Grade 4 Grade 8 Grade 12 Resource Yes No Yes No Yes No Computer at home Charter 90% 10% 94% 6% 90% 10% Traditional 89% 11% 93% 7% 94% 6% Magazines in home Charter 48% 31% 54% 32% 50% 45%* Traditional 54% 26% 60% 30% 61% 35% Newspaper at home Charter 19% 43% 36% 43% 31% 58% Traditional 26% 38% 33% 45% 38% 54% Table reads: At grade 4, 90% of charter school students reported having a computer at home. *The difference in percentages between charter schools and traditional schools is statistically significant at the .05 level.

Several findings can be drawn from table 10, which focuses on the availability of books in the home:

• At grade 4, students from traditional and charter schools reported having similar numbers of books at home.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 34

• At grade 8, a significantly higher percentage of students in traditional schools reported having only 0 to 10 books in their home. A greater percentage of charter schools students reported having 26-100 books, thought the difference is not significant.

• At grade 12, a significantly greater share of traditional public school students than of charter school students reported having more than 100 books in their home.

Table 10. Percentage of students choosing various responses to the question, “About how many books are there in your home?” 2010

0-10 books 11-25 books 26-100 books More than 100 Grade 4 Charter 11% 20% 33% 37% Traditional 12% 22% 35% 31% Grade 8 Charter 9% 20% 43% 29% Traditional 15%* 23% 34% 28% Grade 12 Charter 19% 34% 33% 14% Traditional 12% 20% 37% 30%*

Table reads: At grade 4, 11% of charter school students reported having 0 to 10 books at home. *The difference in percentages between charter schools and traditional schools is statistically significant at the .05 level.

Finally, table 11 addresses the extent to which students discuss their school experiences with family members. Responses from students in charter and traditional public schools who took the civics assessment were similar in terms of how often they discussed what they studied in school with someone at home.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 35

Table 11. Percentage of students choosing various responses to the question, “How often do you discuss things you have studied at school with someone at home?” 2010

Never or hardly ever Every few weeks Once/week 2-3 times/week Every day Grade 4 Charter 17% 17% 11% 21% 34% Traditional 21% 14% 13% 20% 33% Grade 8 Charter 19% 20% 15% 25% 21% Traditional 23% 19% 17% 22% 19% Grade 12 Charter 24% 20% 18% 18% 21% Traditional 19% 19% 20% 24% 18%

Table reads: At grade 4, 11% of charter school students reported having 0 to 10 books at home.

Conclusions about Further Research from the NAEP Analysis

Our analysis of NAEP data should be seen as a “first cut” at examining the differences in civic education between charter and traditional public schools. Many key questions cannot be answered because of holes in the NAEP data. Even when we found statistically significant differences between charter and traditional schools, we had difficulty understanding the implications of these differences because of limitations in the NAEP data. A particularly frustrating hole is the lack of disaggregated NAEP civics data for urban charter schools. Because most charter schools are located in urban areas and serve the diverse student populations more commonly found in cities, it would seem to be more appropriate to compare charter and traditional public schools in urban areas rather than comparing a representative sample of all charter and all traditional schools in the nation. It is not clear whether we would find the same patterns if urban charter schools were compared with urban traditional public schools. Addressing these limitations would be an important part of a future research agenda.

The data that were available showed several similarities between these two school sectors. Overall, achievement in civics was similar for students in charter and traditional public

Center on Education Policy Graduate School of Education and Human Development The George Washington University 36

schools. No significant differences were evident in how often students reported studying social studies. For the most part, 4th grade teachers in both sectors placed about the same amount of emphasis on basic civics topics, and similar proportions of teachers in both sectors reported basing their social studies programs on state curriculum standards.

We did find some statistically significant differences between charter and traditional schools—for example, in average NAEP scores for certain student subgroups at particular grades, in the types of civics topics studied by students, and in the types of instructional methods used to teach social studies. Additional research could help clarify the following questions suggested by statistically significant differences in NAEP results:

• Why do some subgroups, such as Hispanic 8th grade males, seem to perform better in civics in charter schools than in traditional public schools?

• Are there indeed differences in how teachers in charter and traditional schools teach social studies, and if so, what aspects of charter schools account for these differences?

We also found some “suggestive” differences between charter and traditional schools that were not statistically significant but that nevertheless raised interesting questions for further research. Examples of questions arising from these suggestive differences include the following:

• Why do average NAEP civics scores in grade 12 appear to be lower for charter schools than for traditional public schools?

• Which teachers—those in charters or traditional schools—have more discretion in how they teach civics, and how does that affect students’ civics achievement?

Our NAEP analysis set us to thinking about several research questions that cannot be answered by NAEP but would be useful to pursue:

Center on Education Policy Graduate School of Education and Human Development The George Washington University 37

• Are there positive aspects of civics instruction in certain charter schools from which educators in traditional public schools can learn?

• Does the way in which civics is taught influence student achievement? Does a more active level of student engagement (debates, role-playing, etc.) have an impact on student achievement?

• Are there differences between charter and traditional public schools in the extent to which they emphasize and value civic education as a vital part of the curriculum?

The Recommendations section near the beginning of this report includes several steps that could be taken to help answer these questions, improve NAEP data, and strengthen the research base in general.

A Review of Other Research

This section synthesizes research on civic education from sources other than NAEP. We initially set out to review the research literature on civic education in charter schools, but found a paucity of studies that focused directly on this topic. Therefore, we expanded our search to include more general studies of civic education in all public and private schools.

The studies we reviewed address four broad issues, each of which is discussed in this section:

• Civic education in charter schools

• Civics achievement and civic engagement in public and private schools

• Instructional practices for improving civic education and engagement in public or private schools

• Civic education and engagement in private (primarily Catholic) schools

Center on Education Policy Graduate School of Education and Human Development The George Washington University 38

Methodology for Literature Review

Our review included empirical studies, literature reviews done by others, and published doctoral dissertations. To find relevant research, we used a variety of online databases, including those focused on education, American studies, political science, and law and government. A total of 12 databases were searched: Academic Search Complete, Education Abstracts, Educational Administration Abstracts, Education Research Complete, Dissertation and Theses Online, ERIC, JSTOR, ProQuest, America, PAIS International, Social Sciences Citation Index, and PolicyFile. The following search terms were used: charter schools, citizenship, civics, civic education, democracy, engagement, history, political engagement, private schools, public schools, school choice, social studies, and United States.

Most of the research summarized below was published in print-based, peer-reviewed journals; the exceptions were dissertations, conference reports, and one book. To supplement the limited information available from these sources, we also reviewed studies and commentary from research groups and nonprofit organizations focused on civic education, such as the Center for Civic Education (CCE), the Center for Information and Research on Civic Learning and Engagement (CIRCLE), and the American Enterprise Institute (AEI) Citizenship program.

Research on Civic Education and Charter Schools

We found very few empirical studies that specifically examined our main topic of interest— civic education in charter schools. A few doctoral dissertations and papers, described below, found higher levels of civics knowledge and engagement among charter school students than among traditional public school students. These studies have limitations, however, that prompt us to interpret their positive findings with caution. For example, the results of the case studies cited below are not generalizable to other schools or districts, and the other studies are narrow in scope. In addition, students who enroll in charter

Center on Education Policy Graduate School of Education and Human Development The George Washington University 39

schools may be different from other public school students in ways that cannot be controlled for in study design.

Buckley and Schneider (2004) studied the effect of charter schools on civic education by comparing students in grades 7 through 12 in charter and traditional public schools in Washington, D.C. The final sample included 165 charter and public school students, who were surveyed about their levels of community service, civic skills, and political tolerance. Overall, the researchers found a statistically significant and positive effect of charter school enrollment on community service and civics skills. In particular, charter school students were more likely to participate in volunteer work and were more likely to take part in debates or discussion. However, they found no statistically significant effect of charter schools on the students’ levels of political tolerance. This study had various methodological issues that suggest a need for caution in interpreting its generally positive conclusions. First, the study had an alarming 81% attrition rate due to parents dropping out of the study or refusing to allow their children to be interviewed; this led to missing data and a significantly lower sample size than originally planned. Second, students who enroll in charter schools are a self-selected group, in that they have more active parents who seek out alternative schools. Students with more active and involved parents may enter charter schools with higher levels of prior civics knowledge and skills, which could ultimately introduce bias into the study.

Doctoral dissertations on civic education in specific charter schools also found that these schools had a positive effect on the level of engagement and knowledge in their students. The findings in these case studies, however, are not generalizable to charter schools overall. One such study by Gordon (2011) examined the Cesar Chavez School for Public Policy, a public charter school located in Washington, D.C. This researcher found that the campus’s whole-school civic education practices developed democratic responsibility in students (measured by their involvement in school decision-making) by exposing them to political activities in the school. Hinton (2010) analyzed citizenship at HOPE Academy, an urban K-8 charter school in Indiana serving predominantly low-income African American

Center on Education Policy Graduate School of Education and Human Development The George Washington University 40

students. Interviews indicated that although the school lacked an explicit citizenship education agenda, teachers employed other methods—such as classroom decor, school rules, and uniforms—to promote ideals of good citizenship. Additionally, the school sought to foster students’ racial and cultural identity as a way to help them define their own notions of citizenship and increase their levels of engagement.

Some researchers have proposed that by emphasizing order and discipline, charter schools—as well as some private and alternative schools—create non-threatening environments that allow students to feel more secure in discussing political and social ideas (Wolf, 2007; Lake & Miller, 2012). Others suggest that the flexibility afforded to charter schools enables them to design missions, curricula, culture, and practices that promote elements of citizenship (Smith, 1998). There is a lack of empirical research, however, that directly examines the relationship between charter schools’ environment and civics knowledge and engagement.

Civics Achievement and Civic Engagement in Public and Private Schools

Studies in addition to NAEP have used different assessments or surveys to gauge civics knowledge and civic engagement. Some of these studies were conducted more than a decade ago, while others were much more limited in scope than NAEP and were not representative of all schools or students.

The 1999 International Association for the Evaluation of Educational Achievement (IEA) compared the civic achievement of 14-year-old students from both public and private schools in the U.S. and 27 other participating countries (Baldi et al., 2001). The study looked at students’ civics knowledge, civic attitudes, and civic engagement. The U.S. ranked seventh in performance among the participating nations. According to the IEA assessment, students in most countries had an understanding of fundamental democratic values and institutions. Students who studied social studies in school almost every day outperformed students that studied social studies once or twice a week or less frequently. Aside from

Center on Education Policy Graduate School of Education and Human Development The George Washington University 41

voting, students were skeptical about traditional forms of political engagement, but many were open to other types of civic involvement, such as collecting money for social causes. The study also found that students with the most civics knowledge were also those most likely to be receptive to participating in civic activities.

Other studies of civics achievement at the high school level have concluded that many students are unable to answer basic history and civics questions and lack understanding of the American governmental system (Niemi & Junn, 1998; Ross, 2000). Feinberg and Doppen (2010) administered the USCIS Naturalization Test to high school students in Ohio and Georgia. The students generally fared poorly on the test questions about U.S. government, history, and geography, and many failed to correctly answer questions about the political system and major American historical events.

A study of trends in students’ civic engagement by Syvertsen and colleagues (2011) analyzed a 30-year span (1976-2005) of cross-sectional data from Monitoring the Future, a survey of levels of engagement of public and private high school seniors. The authors’ measure of civic participation was based on whether study participants had already done or planned to do the following activities: “write to public officials, give money to a candidate or cause, and work in a political campaign” (p. 587). In addition, two other measures asked students about participating in lawful demonstrations and boycotting specific products and stores. Overall, the researchers found general declines in adolescents’ civic participation, although there were slight increases in community service participation. The findings also showed that youth with aspirations to graduate from two- or four-year colleges were more likely to engage in civic activities than youth with no college aspirations.

A recent report released by Educational Testing Service (ETS) analyzed levels of civic engagement of American adults using the Civic Engagement Index, which measured voting and volunteer behavior (Coley & Sum, 2012). This study showed that civic engagement varied across levels of educational attainment, age, and income; rates of engagement were

Center on Education Policy Graduate School of Education and Human Development The George Washington University 42

positively correlated with higher education and income levels and increases in age. According to the study’s authors, these findings suggest a need for schools or the larger U.S. society to focus on “strengthening of core democratic and civic practices among young, less-educated, and less-affluent members of our society” (p. 24).

Various studies have analyzed possible reasons for gaps in civics knowledge or civic engagement among racial/ethnic minority and disadvantaged students. One such study by Torney-Purta, Barber, and Wilkenfeld (2007) examined results for Latino and non-Latino 9th graders from the aforementioned 1999 IEA Civic Education Study. Their quantitative findings showed that non-Latino students had significantly more civics knowledge, were more likely to expect to vote, and were less likely to hold positive attitudes toward immigrant rights than were Latino students. The researchers attributed the gaps between the two groups to individual- and school-level characteristics, such as the level of political engagement in the home, classroom environment, and teachers’ pedagogical practices. For example, the gap between Latino and non-Latino students in their expectation to vote may be a result of the absence of civics instruction in schools that serve a majority of Latino students. The study suggested that schools play an important role in “forming identities, fostering attitudes, and providing instruction to effectively develop knowledge and skills” (p. 122).

Some researchers have proposed that differences in family background characteristics or community opportunities contribute to disparities in civics achievement and civic participation. Flanagan and Levine (2010) suggested that many disadvantaged students come to the classroom with lower levels of existing civics skills and knowledge due to unequal opportunities. Other studies have shown that students who are not college-bound or who attend school in high-poverty districts have fewer opportunities to develop their civic participation and skills (Kahne & Middaugh, 2008; Hart & Atkins, 2002). Furthermore, poor and minority students often lack opportunities to engage in civic activities in their community and school. Lastly, many immigrant students often face the

Center on Education Policy Graduate School of Education and Human Development The George Washington University 43

added challenge of a language barrier, which prevents them from actively participating in civic and political activities (Torney-Purta, Barber, & Wilkenfeld, 2007).

Recent studies have shown that civic education for disadvantaged students in public and private schools can have a large effect in compensating for these students’ lack of prior civics knowledge and skills. By contrast, civic education may have a smaller effect on skill development than does family and home environment for students who have alternative opportunities to develop their civic skills, such as living in an active political home environment (Comber, 2009; Flanagan & Levine, 2010; Gainous & Martens, 2012). Comber (2009) analyzed the effects of civics coursework on civics skills, as measured by students’ political interpretation skills, and found that the effects were stronger for African-American and Latino students than for white students, mainly due to the minority students’ lack of prior knowledge.

Instructional Practices for Improving Civic Education and Engagement

Another strain of education research has focused on identifying key practices aimed at improving civic education in American public and private schools. These studies have detected a few curricular and instructional strategies that improve civics knowledge and engagement. However, the findings are based on specific case studies, and the practices they highlight are working in unique contexts. The findings are not generalizable because the types of curricula found in the case study schools are not typically implemented in schools throughout the U.S.

Initial research in the 1960’s found that school-based civic education was ineffective at encouraging participation and knowledge (Litt, 1963; Langton & Jennings, 1968). More recent research, however, has identified the important role of schools in fostering civic engagement (Niemi & Junn, 1998; Galston, 2001). Youniss (2011) found that certain school practices, such as promoting classroom discussions, offering student government and other

Center on Education Policy Graduate School of Education and Human Development The George Washington University 44

extracurricular activities, and incorporating service learning into the curriculum, can build students’ knowledge of history and government and encourage their civic engagement.

Studies by the Center for Civic Education and by outside evaluators have focused on the effects of courses for public and private school students developed by this Center. One such course, We the People: The Citizen and the Constitution, aims to promote “civic competence and responsibility” through critical thinking, interactive strategies, and activities such as simulated congressional hearings and national competitions (Eschrich, 2010, p. 1). According to the most recent study of the effects of the course on students’ political knowledge, students and alumni of We the People knew more about American government and institutions than the general public (Owen, 2011).

Another study published by the Center for Civic Education compared the constitutional and civics knowledge and the civic skills and disposition of students who participated in We the People with those of nonparticipants (Turnbull, Root, Billig, & Jaramillo, 2007). The study, which included 735 matched comparison high school students from five states (Colorado, Indiana, Michigan, New York, and Washington), focused on knowledge of American history and the Constitution, and the perceived importance of participating in civic activities such as voting, campaign work, and protests. We the People students scored significantly higher on civics knowledge and civic skills and disposition, the study noted.

To identify practices for increasing the civic engagement of disadvantaged and minority youths, Journell and Castro (2011) conducted a case study of civics classes taught by a particular teacher at a Chicago public high school serving a predominantly Hispanic and African American population. This teacher used “culturally relevant” instructional practices that focused on issues of race and immigration to teach students about the American political system. The researchers concluded that incorporating current, real-life topics into civic education helped to put these topics in context and narrow the gap between civics narratives in textbooks and traditional curricula and the everyday experiences of students. According to this study, students’ interest in the subject matter

Center on Education Policy Graduate School of Education and Human Development The George Washington University 45

increased as they questioned claims made in textbooks by comparing it to their own life experiences. The researchers noted, however, that these practices are not typical.

Another case study by Pope, Stolte, and Cohen (2011) sought to shed light on best practices for civic engagement. The study focused on Generation Citizen, a student-based, action- centered program that encouraged project-based learning to boost civic engagement and knowledge among underrepresented youth. The program, which was implemented in schools throughout Boston, Providence, and New York City, was shown to increase the use of best practices, such as studying government and current events and participating in service learning and real-life democratic processes. In addition, the study found that these action-centered activities promoted reflection and self-efficacy in students.

Lastly, a study conducted by Feldman, Pasek, Romer, and Jamieson (2007) examined the Student Voices curriculum, which was implemented in 22 Philadelphia-area high schools. The analysis showed that specific components of the program—particularly class discussions, political community projects, and use of the Internet—increased participants’ tendency to follow and discuss politics, broadened their knowledge of the political system, and improved political efficacy compared with students in a traditional civics curriculum. Program gains were equal for white and minority urban students, which the study authors saw as an indication that Student Voices and similar curricula can help narrow the civics gap.

Civic Education and Engagement in Private Schools

Some research has focused on the role of civics knowledge and civic engagement in private schools, particularly Catholic schools (Belfield, 2004; Campbell, 2008; Wolf, 2007). Studies have shown that attendance at Catholic schools has a positive effect on both of these outcomes, although it has a marginally larger effect on engagement (Belfield, 2004; Campbell, 2008). Using the U.S. National Household Education Survey of 1999, Belfield (2004) found that attending a Catholic or private independent school had a strong positive

Center on Education Policy Graduate School of Education and Human Development The George Washington University 46

effect on community service participation, civic skills, civic confidence, political knowledge, and political tolerance. Further, the analysis concluded that these schools had higher levels of civic education than traditional public schools.

Campbell (2008) also compared measures of civic education (specifically, participation in community service, civic skills, political knowledge, and political tolerance) between private and public school students. This study found that Catholic school students had higher degrees of voluntarism, civic skills, political knowledge and tolerance, while students in the private secular schools showed higher levels of tolerance than their public school counterparts. Both Belfield (2004) and Campbell (2008) concluded that since Catholic schools integrate community service and volunteerism into the school curriculum, it is not surprising to see a positive effect of Catholic school attendance on levels of civic engagement.

The Lack of Research on Civic Education

The purpose of this literature review was to examine civic education in charter schools, but there was a general lack of research on this specific topic. Indeed, we found little research on civic education period—whether in traditional public, charter, or private schools. As discussed in the recommendations for this report, this lack of research leaves the nation at a disadvantage in understanding the status of civics knowledge and instruction and civic engagement in our youth.

Although several studies concluded that charter schools led to higher levels of civics knowledge and engagement, the limitations of these studies suggest a need for caution in interpreting their findings. Limitations are also apparent in the case studies of instructional practices aimed at fostering civics knowledge and civic engagement in minority and disadvantaged students. The findings from this research are not generalizable because these approaches are not representative of typical civics curricula (Journell & Castro, 2011). Although these studies looked at a range of classroom practices, the researchers

Center on Education Policy Graduate School of Education and Human Development The George Washington University 47

failed to isolate the most effective practices out of the variety of options, particularly when examining which methods increase civics knowledge. Further research into specific classroom and curricular practices is needed to help practitioners determine which changes should be encouraged in public school civics courses.

The Recommendations section near the beginning of this report lays out steps that might begin to answer some of the most pressing issues raised in our literature review.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 48

References

Baldi, S., Perie, M., Skidmore, D., Greenberg, E., & Hahn, C. (2001). What democracy means to ninth-graders: U.S. results from the international IEA civic education study (NCES 2001– 096). Washington, DC: U.S. Government Printing Office.

Belfield, C. R. (2004). Democratic education across school types: Evidence for the U.S. from NHES99. Education Policy Analysis Archives, 12(43). Retrieved from http://epaa.asu.edu/epaa/v12n43/.

Buckley, J., & Schneider, M. (2004, March). Do charter schools promote student citizenship? Paper presented at the Annual Meeting of the Midwest Political Science Association, Chicago, IL.

Campbell, D. E. (2008). The civic side of school choice: An empirical analysis of civic education in public and private schools. Brigham Young University Law Review, 2008(2), 487-523.

Center on Education Policy. (2008, February). Instructional time in elementary schools: A closer look at changes for specific subjects. Washington, DC: CEP.

Center on Education Policy. (2010). State test score trends through 2008-09, part 1: Rising scores on state tests and NAEP. Washington, DC: CEP.

Coley, R. J., & Sum, A. (2012). Fault lines in our democracy: Civic knowledge, voting behavior, and civic engagement in the United States. Princeton, NJ: Educational Testing Service.

Collins English Dictionary. (2012). Definition of “civics.” http://www.collinsdictionary.com/dictionary/english/civics?showCookiePolic y=true

Comber, M. (2009). Learning to participate: The effects of civic education on racial/ethnic minorities. National Political Science Review, 12, 179-194.

Eschrich, D. (2010). We the people: The citizen and the Constitution, 2010 national finalists’ knowledge of and support for American democratic institutions and processes. Calabasas, CA: Center for Civic Education.

Feinberg, J. R., & Doppen, F. H. (2010). High school students’ knowledge and notions of citizenship. The Social Studies, 101, 111-116.

Feldman, L., Pasek, J., Romer, D., & Jamieson, K. H. (2007). Identifying best practices in civic education: Lessons from the student voices program. American Journal of Education, 12(1), 26-37.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 49

Flanagan, C., & Levine, P. (2010). Civic engagement and the transition to adulthood. Future Child, 20(1), 159-179.

Gainous, J., & Martens, A. M. (2012). The effectiveness of civic education: Are “good” teachers actually “good” for all students? American Politics Research, 40(2), 232-266.

Galston, W. A. (2001). Political knowledge, political engagement, and civic education. Annual Review of Political Science, 4, 217-234.

Godsay, S., Henderson, W., Levine, P., & Littenberg-Tobias, J. (2012). State civic education requirements. Medford, MA: Center for Information and Research on Civic Learning and Engagement (CIRCLE).

Gordon, P. J. (2011). Building voice, taking action: Experiences of youth from a civic focused school. (Published doctoral dissertation). Harvard Graduate School of Education, Cambridge, MA.

Hart, D., & Atkins, R. (2002). Civic competence in urban youth. Applied Developmental Science, 6(4), 227-236.

Hinton, H. (2010). No one rises to low expectations: Citizenship education in an “urban” charter school. (Published doctoral dissertation). Purdue University, West Lafayette, IN.

Journell, W., & Castro, E. L. (2011). Culturally relevant political education: Using immigration as a catalyst for civic understanding. Multicultural Education, 18(4), 10-17.

Kahne, J., & Middaugh, E. (2008). Democracy for some: The civic opportunity gap in high school (Working Paper 59). College Park, MD: CIRCLE.

Lake, R., & Miller, C. (2012). Strengthening the civic mission of charter schools. Washington, DC: AEI Program on American Citizenship.

Langton, K. P., & Jennings, M. K. (1968). Political socialization and the high school civics curriculum in the United States. The American Political Science Review, 62(3), 852-867.

Litt, E. (1963). Civic education norms and political indoctrination. American Sociological Review, 28, 69-75.

Niemi, R. G., & Junn, J. (1998). Civic education: What makes students learn. New Haven, CT: Yale University Press.

Owen, D. (2011). We the people and political knowledge. Woodland Hills, CA: Center for Civic Education.

Pope, A., Stolte, L., & Cohen, A. K. (2011). Closing the civic achievement gap: The potential of action civics. Social Education, 75(5), 265-268.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 50

Ross, E. W. (2000). Creating a civic-minded political culture: What does the civics NAEP mean for social studies education? Theory and Research in Social Education, 28(1), 6-10.

Rubin, B. C. (2007). “There’s still not justice”: Youth civic identity development amid distinct school and community contexts. Teachers College Record, 109(2), 449-481.

Smith, S. (1998). The democratizing potential of charter schools. Educational Leadership, 56(2), 55-58.

Syvertsen, A. K., Wray-Lake, L., Flanagan, C. A., Briddell, L., & Osgood, D. W. (2011). Thirty- year trends in U.S. adolescents’ civic engagement: A story of changing participation and educational differences. Journal of Research on Adolescence, 21(3), 586-594.

Torney-Purta, J., Barber, C. H., & Wilkenfeld, B. (2007). Latino adolescents' civic development in the United States: Research results from the IEA civic education study. Journal of Youth and Adolescence, 36(2), 111-125.

Turnbull, J., Root, S., Billig, S., & Jaramillo, D. (2007). We the people: The citizen and the Constitution. Denver, CO: RMC Research Corporation.

U.S. Department of Education, National Assessment of Educational Progress. (2011a). Civics assessment time by content area [Web page]. Retrieved from http://nces.ed.gov/nationsreportcard/civics/timebycontent.asp

U.S. Department of Education, National Assessment of Educational Progress. (2011b). The nation’s report card: Civics 2010. Retrieved from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2011466

U.S. Department of Education, National Center for Education Statistics. (2012a). Charter school enrollment. The Condition of Education. Retrieved from http://nces.ed.gov/programs/coe/indicator_cse.asp

U.S. Department of Education, National Center for Education Statistics. (2012b). Characteristics of elementary and secondary schools. The Condition of Education. Retrieved from http://nces.ed.gov/programs/coe/indicator_ces.asp

U.S. Department of Education, National Center for Education Statistics. (2012c). Racial- ethnic enrollment in public schools. The Condition of Education. Retrieved from http://nces.ed.gov/programs/coe/indicator_1er.asp

Wolf, P. J. (2007). Civics exam: Schools of choice boost civic values. Education Next, 7(3), 66- 72.

Youniss, J. (2011). Civic education: What schools can do to encourage civic identity and action. Applied Developmental Science, 15(2), 98-103.

Center on Education Policy Graduate School of Education and Human Development The George Washington University 51

Credits and Acknowledgments

The NAEP analysis in this report was researched and written by Naomi and Victor Chudowsky, CEP consultants. Nanami Yoshioka, CEP’s graduate research assistant, and Jennifer McMurrer, CEP’s research associate, conducted and wrote the review of other research. Nancy Kober, a CEP consultant, assisted with writing and editing the entire report. Maria Ferguson, CEP’s executive director, and Diane Stark Rentner, CEP’s deputy director, provided advice and assistance on the report.

Based in Washington, D.C., at The George Washington University’s Graduate School of Education and Human Development and founded in January 1995 by Jack Jennings, the Center on Education Policy is a national independent advocate for public education and for more effective public schools. The Center works to help Americans better understand the role of public education in a democracy and the need to improve the academic quality of public schools. We do not represent any special interests. Instead, we help citizens make sense of the conflicting opinions and perceptions about public education and create the conditions that will lead to better public schools.

The Center on Education Policy receives nearly all of its funding from charitable foundations. We are grateful to the Spencer Foundation for supporting this project. The statements made and views expressed are solely the responsibility of the Center.

© Center on Education Policy, December 2012

Center on Education Policy Graduate School of Education and Human Development The George Washington University 2140 Pennsylvania Avenue NW, Suite 103 Washington, D.C. 20037 Ph: 202-822-8065 Fax: 202-994-8859 E-mail: [email protected] Web: www.cep-dc.org

Center on Education Policy Graduate School of Education and Human Development The George Washington University