How Accountability Impacts High School Math and English Departments
Subject Matters:
How Accountability Impacts High School Math and English Departments
Dana Holland, Betheny Gross, and Joy Anderson
Consortium for Policy Research in Education
University of Pennsylvania
A paper prepared for the
Annual Conference of the American Association of Educational Research
Montreal, Canada
April 2005
I. Research Focus
High schools are increasingly coming into focus as the objects of federal and state accountability systems. At the recently concluded National Governors’ Association summit, both business and political leaders advocated for restructuring and sustained change for high schools, putting an end to the “culture of educational complacency” as one governor put it (Olson 2005). However, the differentiated organization of high schools presents special challenges to standards-based accountability, which assumes that the school is the unit of improvement (Fuhrman 1999). High schools are large and complex, and whereas teachers in comparatively homogeneous lower grades might see the school as their primary reference group, high school teachers tend to regard themselves foremost as members of subject matter departments (Johnson 1990). These “invisible structures,” as Siskin (1994) has called departments, create strong organizational boundaries within high schools, influencing the professional identity and capacity for action of the teachers within these boundaries.
This study examines the overall and differential impact of standards-based accountability on the goals and experiences of teachers in high school math and English departments, subjects drawing the most attention in accountability systems. One of the hallmarks of the new accountability is an exchange of flexibility for results—policy makers determine targets, while teachers, schools, and districts are left to determine the response. The study focuses on how departmental structure figures into high school responses to accountability in different state contexts. It does not link variation in departmental responses to changes in student achievement; it is instead an exploratory investigation of both how departmental structure influences response to accountability and how accountability seems to be influencing departmental structure. In considering how policy interacts with the organizational structure in high schools, we assume that organizational differentiation within high schools and the nature of accountability has the potential to impact teachers in the same school differently. Explanations for these differential responses are therefore argued here to follow from the interactions among the specific characters of departments, the designs of state accountability systems, and more or less inherent features of different subject matters.
This work is part of a larger multi-year study of high schools being conducted by the Consortium for Policy Research in Education (CPRE), with funding provided by the Institute for Educational Sciences. This larger study investigates strategies for instructional improvement in American high schools, examining how below-average performing high schools incorporate their states’ accountability goals into their own, identify their challenges, search for improvement strategies, and generally respond to the gap between their current levels of achievement and external expectations.
II. Relevant Theoretical and Empirical Research
Importance of Departments in High Schools
The “egg crate” model of high schools as places where privacy norms of professional practice prevail (Lortie 1975; Jackson 1968), has been tempered with scholarship showing the mediating influence of departments on teachers’ work and the social organization of the high school (Bidwell and Yasumoto 1999; Johnson 1990; Little 2002; Siskin 1994; McLaughlin and Talbert 2001; Rowan 1990). Although departments show considerable variation in their roles, level of cohesion and responsibility, research has consistently shown that teachers tend to identify their subject-based colleagues as their core professional group with whom they share knowledge, goals, and experiences. Siskin (1994) found departments in one California high school to “comprise different worlds, exhibiting different cultures, and controlling key decisions about resources, professional tasks, and careers.” Although the “collegial focus” of departmental social organization has been found to provide structural and normative capacity for collegial control of instruction (Bidwell and Yasumoto 1999), it is unclear how accountability systems, which impose various levels and kinds of control on curriculum and therefore on instruction, affect collegial curricular activities.
Typologies of High School Departments
Typologies have been developed to characterize departments by the key dimensions influencing their efficacy as organizational units.[1] McLaughlin and Talbert (2001, 62) differentiate between strong and weak departments, defining weak departments as typified by individuated values and beliefs, in contrast to strong departments in which goals and beliefs are held in common. In weak departments, traditional practice predominates, and any innovation that happens occurs in isolation. Strong departments can support either traditional community that enforces socialization to traditional practice, or communities of practice, where organizational learning and collaboration is built into the group’s ethos. In this scheme, strong high school departments can be reactionary or reformist, with capacity to mobilize for resistance or for change in core practices.
Siskin (1994, 100) identifies two key dimensions of departments, inclusivity, which indicates how encompassing is the collectivity, and commitment, which indicates how widely goals and purposes are shared. These two dimensions inform a four-part classification:
Bundled
Low CommitmentHigh Inclusion /
Bonded
High CommitmentHigh Inclusion
Fragmented
Low CommitmentLow Inclusion /
Split
High CommitmentLow Inclusion
Bonded departments, which are akin to McLaughlin and Talbert’s professional learning community, are indicated by regular collaboration among department members with efforts communally directed toward reaching departmental goals. In Siskin’s (1994) study of three high schools in two states, California and Michigan, conducted in the early 1990s, bonded departments were rare. Bundled departments, where teachers are part of a cohesive community but hold individual goals and work generally in isolation, were much more commonly represented in Siskin’s sample. In bundled departments, teachers identify with the group and coordinate when necessary, but privacy norms militate against formal collaboration in curriculum and instruction. In fragmented conditions departments are organized units in name only, where teachers work in isolation, goals are atomized, and individuals are largely left to identify and solve their own problems. Split departments exhibit breaches of one sort or another, with factions of teachers aligned along the lines of age, seniority, race, gender, or some other identity characteristic.
Despite their rarity, strong learning communities, which Siskin would call bonded departments, are lauded by McLaughlin and Talbert as being most beneficial to students. They contend that communities of practice minimize the “instructional lottery” for students, in which classroom experiences are determined by the individual proclivities of the teacher to whom a student happens to be assigned, rather than by the concerted effort of a unified, improvement-oriented collective. Notably, scholars in the field of organizational learning have found that communities of practice are associated with innovation in the workplace (Brown and Duguid 1991), confirming McLaughlin and Talbert’s argument that this normative type is associated with capacity to improve.
The Consequences of Subject Matter on Departments
The focus on departments has been important in redressing tendencies to view the high school as a homogeneous unit akin to elementary schools. However, some argue that departments themselves can be substantively differentiated. While acknowledging the influence of “subject perspectives,” Ball and Lacey (1980) argue against a model of departments that constitute undifferentiated knowledge sub-cultures. In a study of four secondary schools in Britain they found that English departments supported multiple subject paradigms within one department. Subjects, this work implies, support some degree of internal flexibility. But what consequences does this have for curricular activities, and are all high school subjects similar disposed to internal diversity?
Stodolsky and Grossman (1995) take up these issues in an investigation of how teachers’ perceptions of the qualities of five different subjects create conceptual contexts for teachers that vary from field to field. In turn, they then examine how key features of these subjects affect curricular coordination and control. They identify five features of subject matter: definition (how clear are the boundaries of a field); scope (the extent a school subject is homogenous or a composite of different fields); sequence (how much prior learning is perceived as prerequisite to later learning); dynamism (how static or dynamic is the subject); and elective or required status of the subject. Based on survey data collected from teachers in 16 schools in Michigan and California in the early 1990s,[2] they found, as expected, that math and foreign language teachers see their subjects as significantly more defined, more sequential, and less dynamic than did teachers in English, science or social studies. They also found that the qualities that teachers associate with their subjects have consequences for how they approach their curriculum. While almost all teachers claimed near total control over instruction, the relatively more sequential subjects, most notably math and foreign language, produced greater curricular and content coordination among teachers. Less well-defined subjects, namely English, science, social studies, prompted less consensus among teachers in those areas. Press for coverage (perceived pressure to cover a defined scope and sequence of curriculum) was similarly highest for math and foreign language, and lowest for science and social studies, with English in the middle. This research suggests that features of math and English as subjects have opposite kinds of consequences for the social organization of the high school department.
The Policy Environment
Research on high school departments usefully highlights the specific organizational conditions in which teachers work. Research on subject matter indicates how content too can introduce curricular constraints and organizational conditions. The policy environment introduces yet another factor that intersects with organizational structure and subject matter conditions to influence teaching and learning in high schools. While the policy context is regularly considered in studies of high school departments, the passage of No Child Left Behind, with its heightened emphasis on state-wide standardized testing and uniform high standards, ushered in a novel task for high schools: preparing all students to achieve at a minimum standard (Siskin 2003).
The link between high school departments and present-day accountability contexts has not been fully examined. Previous research suggests how the current push for standards and standardized testing is likely to impact departments as organizational units housing subject matter specialists. Referencing their research as anchored in the standards-setting movement of the 1980s, Archbald and Porter (1994) compared high school teachers’ control over curriculum in two subjects, math and social studies. They found that external tests and curriculum guidelines were viewed as more influential by math than social studies teachers. However, the two groups of teachers did not differ in their perceptions of control over content and pedagogy. Despite stiffer external controls, math teachers’ sense of control over pedagogy (instruction, homework, and student achievement standards) was in fact higher than that of social studies teachers. This suggests the possibility of a department effect, propagating belief in the consonance between district level curricular control and the goals and efforts of department members. Siskin (2003) in a study of high schools and accountability in four states found that high schools in states with weak or moderate accountability showed “dramatic differences” among the schools’ accountability responses in comparison to schools in states where sanctions are stronger and systems have been in place longer. She observed some instances of teacher engagement in substantive and sustained conversation about teaching to the new standards. However, this happened in departments, and very few departments in her sample were organized to prompt or sustain these conversations.
Some researchers argue that policy design should take both departments and subject matter in high schools more seriously, building mechanisms into the accountability system to encourage professional interaction. Based on data collected in the early 1990s, McLaughlin and Talbert (2001) conjecture that it was no accident that the strong communities of practice found among the schools in their sample were located in California, where “new standards for teaching and learning were embodied in core subject frameworks, and where teacher professional development initiatives and subject-area networks were growing.” Siskin (1994) predicts that without overt efforts to change curriculum and instruction, “subject-by-subject,” that is within departments, that the impact of standards-based accountability on student achievement in high schools will be negligible.
III. Hypotheses and Expectations Based on Previous Research
1) Research on departments suggests that bonded departments, or strong communities of practice, would be most likely to mount a coordinated response to accountability, especially if the goals of accountability have been fully incorporated into the department’s own. Given their strong sense of identity, these departments would likely also hold other goals, and members would likely not present themselves as fully captured by accountability demands. In so far as the accountability system makes the district or school principal into an agent in the implementation of accountability prescriptions, we would expect them to be more of a presence in departmental matters regardless of departmental type.
2) Research on departments suggests the possibility of differential responses to accountability among departments in the same school. Following from argument that departments themselves can be differentiated identity units, we also hypothesize the possibility of differential responses within departments. We expect that the specific design attributes of accountability systems might have different effects on teachers depending on whether or not they are working with courses, grade levels, or students targeted by the accountability system.
3) Research on the relation of subject matter to curricular activities leads us to predict substantial differences between math and English regardless of accountability context. The relatively more defined, sequential, and static features of math as a subject contrast with English. Because of these subject matter features, math teachers exhibit more consensus and coordination around curriculum and more coordination in curricular activities than English teachers. However, in so far as accountability system design and pressure recasts English as focused primarily on language and communicationskills, then we expect that English as a subject might take on some of the features that Stodolsky and Grossman (1995) identify for foreign language, which are comparable with math. Where math and English teachers appear to be responding similarly to accountability, it might not be that there is a school level response in operation, but instead that English as a subject has become and been accepted as being more “math-like.”
IV. Methodology
The larger study from which this paper draws is based on a nested sample of 48 schools in 36 districts in six states, selected to include two weak accountability states (Pennsylvania and Michigan) and four strong accountability states (North Carolina, Florida, New York, and California). Using discussions of the strength of state accountability systems conducted by Goertz and Duffy (2001) and Carnoy and Loeb (2004), a strong accountability state was defined as one that had sanctions in place for schools and students, while a weak accountability state had no sanctions at the local level for either schools or students. The sample for the full study included relatively low performing schools, since they are the primary targets of state accountability policy, and represented a variety of school contexts, in terms of urbanicity, socio-economic status of the community, and diversity of the student body. A stratified random sampling frame was developed using 1999-2000 school achievement and context data out of which eight schools in each state were selected. Fieldwork was carried out during the 2002-03 school year and involved structured interviews with a set of school and district representatives, including formal school and district leaders, curriculum specialists, math, English, and foreign language department heads, and two teachers each from math and English. The larger study included data collected about goals, challenges, decision-making practices, informational search strategies, perceptions about accountability, and improvement efforts underway in the school. Interviews were tape recorded and fully transcribed.
For this paper, we conducted school level analysis of data collected for two high accountability states (North Carolina and Florida) and two low state accountability states (Pennsylvania and Michigan). Specifically, we analyzed the goals of math and English departments using the site summaries prepared by researchers conducting the fieldwork for each of the 31 schools we studied in these states.[3] Although these data and the analysis are qualitative, we used a data analysis software program, Survey-pro, to aggregate the data and run simple cross-tabulations. For two states, Pennsylvania and North Carolina, we then went back to the original 89 interviews to examine in more detail how and under what conditions high school departments mount organized responses to accountability. Our analysis expressly compares high and low accountability contexts, with awareness of important design variations within these categories, as well as between math and English.
The methodology used here has strengths and weaknesses. Our four-state sample included 192 interviews with teachers, and these interviews provided rich information. However, the cross-sectional study design included only one visit to each school, so we were not able to observe departmental organization or to determine how responses to accountability occurred in real time. In addition, we interviewed three teachers each in math and English departments, effectively building up a view of the department from the responses shared by these three teachers. Although our sampling frame called for interviews with teachers working in different grade levels and we were able to triangulate responses, a sub-sample of three teachers clearly better represents small and medium size departments more than large departments and this is a weakness in our design.