Information to Users

Home , Fox Classics, Sex Ed (The Office)

INFORMATION TO USERS

The most advanced technology has been used to photograph and reproduce this manuscript from the microfilm master. UMI films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer.

The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrougb, substandard margins, and improper alignment can adversely affect reproduction.

In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion.

Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand corner and continuing from left to right in equal sections with small overlaps. Each original is also photographed in one exposure and is included in reduced form at the back of the book.

Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6" x 9" black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order.

University Microfilms International A Bell & Howell Information Company 300 North Zeeb R oad Ann Arbor, Ml 48106-1346 USA 313/761-4700 800/521-0600

Order Number 9031186

Policy and conversation: A case study of the implementation of a state testing policy in a school district

Zajano, Nancy Carol, Ph.D.

The Ohio State University, 1990

UMI 300 N. Zecb Rd. Ann Arbor, Ml 48106

POLICY AND CONVERSATION: A CASE STUDY OF THE

IMPLEMENTATION OF A STATE TESTING POLICY

llN A SCHOOL DISTRICT

DISSERTATION

Presented in Partial Fulfillment of the Requirements for

the Degree Doctor of Philosophy in the Graduate

School of the Ohio State University

Nancy Carol Zajano, B.A., M.A.

*****

The Ohio State University

1990

Dissertation Committee: Approved by

C. M. Galloway, Co-Chair

B. L. Mitchell, Co-Chair Adviser R. W. Backoff College of Education R. Donmoyer 1

Nancy Carol Zajano

1990 To Mike,

Rachel, Maureen, and Emily

Brad

"Bill" and the Educators in "Foster" ACKNOWLEDGMENTS

The work of this study has been sustained by many people. The care, concern and ongoing support of my family, friends, and intellectual mentors have allowed me to discover many possibilities within myself during my doctoral studies.

For four years, Brad Mitchell has challenged me to find and use my own voice in academic endeavors. His tireless scholarship and personal courage in the pursuit of transformative ways of teaching, researching, and interacting have served as a powerful role model for my own efforts. I thank him for risking being simultaneously a professional and personal friend, and for balancing the two roles with grace and wisdom.

In my exploration of organisational cultures, Charles

Galloway helped me to understand the role of verbal and nonverbal expressions in defining the relationships between people. I have treasured our conversations on organizational life within the university. For his insights on which words of encouragement are needed when during the process of being a student and becoming a scholar, I am

iii deeply grateful.

As a student of Robert Backoff I was treated to a masterful combination of challenge and support through a thoughtful yet indirect approach to pedagogy. His deep regard for the intellectual possibilities of students brings them to challenge themselves to meet his unspoken but powerful expectations, I thank him for sharing his gifts of mind, spirit, and self in our classes and learning groups,

My thanks to Robert Donmoyer, Donald Sanders, and Patti

Lather for their teaching and examples of methodologically sound interpretive research. Because this particular way of knowing is both personally and professionally rewarding to me, 1 am grateful for their guidance. Robert Donmoyer is especially gifted at shining a flashlight into the dark corners of qualitative research where murky ideas benefit from his illumination.

My friends Sue Rieger and Karen Fellows have made graduate student life a joy. The development of mind, self and voice could not have occurred with better company. As a peer debriefer, Pat Scharer explored with me the different paths and interpretations unfolding in this study. Her insight, wit and encouragement never wavered throughout the process. My friend Deb Voege continues to provide a model of a person who can give to others while maintaining strong

iv goals for herself. To my kindred spirits in Rhode Island,

Sharon Rallis and Doris Donovan, whose emotional support

and intellectual challenges have sustained me for many

years, I extend my deepest appreciation.

The educators in the school district where I conducted

my study opened their buildings, thoughts and feelings to

my probing questions without regard for how the research

might help or hurt them. For their unabashed willingness to

share in my learning, I am deeply grateful. It is hard to

know how to thank "Bill" for allowing me to put him under

my research microscope and question his every action and

interaction. He never flinched at my unrelenting questions;

something my dearest friends have not endured for nearly as

long.

My husband Mike has been my closest friend for over 20

years. During these most recent graduate student years he

never wavered in his care and encouragement. In addition to meeting the physical and emotional needs of our three daughters, he willingly explored whatever idea was

intriguing me at the moment. His love and friendship have been an amazing grace.

To Maureen, Emily and Rachel, for understanding that

although Mom was perpetually "in her hole" in the basement office, she could share in their delightful and unfolding

lives.

v VITA

November 13, 1946 Born, Aurora, Illinois

1968...... B.A., St. Norbert College West De Pere, Wisconsin

1970...... M.A., Roosevelt University Chicago, Illinois

1968-1971 ...... Teacher Chicago Public Schools

1971-1976...... Evaluation Specialist Wisconsin Research and Development Center for Cognitive Learning, University of Wisconsin-Madison

1976-1985...... Research Associate Center for Evaluation and Research, Rhode Island College, Providence, Rhode Island

1982-1985...... Acting Director Center for Evaluation and Research, Rhode Island College, Providence, Rhode Island

1986-1989.... Graduate Research Associate PROBE, The Ohio State University, Columbus, Ohio

1989-1990...... Presidential Fellow The Ohio State University Columbus, Ohio PUBLICATIONS

Zajano, N. C,, & Mitchell, B. (1988). Schoolhouse blues: Tensions among research, policy and practice. Theory into Practice., 27_t 161-168.

Zajano, N. C., Mitchell, B., Schiraldi, F. (1989). Principal policy in Ohio. Columbus, OH: The Ohio State University, Policy Research for Ohio Based Education (PROBE).

Zajano, N. C. (1989). Reading Recovery and ESEA Chapter 1: Issues and possibilities. Columbus, OH: The Ohio State University, Policy Research for Ohio Based Education < PROBE).

Zajano, N. C., & Mitchell, B. (1987). Students "At-Risk'1: Points to ponder for Ohio educational policy. Columbus, OH: The Ohio State University, Policy Research for Ohio Based Education (PROBE).

FIELDS OF STUDY

Major Field: Education

Studies in Policy Research: Dr. Brad Mitchell Dr. Robert Backoff Dr. Robert Donmoyer

Studies in Communication: Dr. Charles Galloway

Studies in Organizational Theory: Dr. Robert Backoff Dr. Brad Mitchell

Studies in' Qualitative Research Dr. Robert Donmoyer Dr. Robert Backoff Dr. Brad Mitchell Dr. Patti Lather

vii TABLE OF CONTENTS

ACKNOWLEDGMENTS...... iii

VITA...... vi

LIST OF TABLES...... xii

CHAPTER PAGE

I. INTRODUCTION...... 1

Background for the Study: Policy Implementation Literature...... 3 Statement of the Problem and Research Questions...... 6 Methodology...... 7 Rationale for the Inquiry...... 9 Delimitations of the Inquiry...... 10 State Testing Policy...... 12 Community and Schools of Foster...... 19 Def initions...... 21 Overview of Chapters Two through Five...... 24

II. THEORETICAL, PHILOSOPHICAL AND RESEARCH BACKGROUND...... 26

Introduction...... 26 Language and Culture...... 28 Social Construction of Reality...... 31 Discourse...... 3 7 Culture, Context and Discourse Rules...... 39 Organization and Culture...... 42 Organizations as Relationships...... 44 Organizational Policies and Commun i cat i on...... 47 Summary and Example from Schools...... 51 Discourse/Conversation...... 55 Policy...... 62 Policy Definitions and Types...... 62 Policy Analysis and Problem Setting...... 67 Policy Implementation...... 72

viii School Reform Policies: Top-down and/or Bottom-up...... 75 State Testing Policies. . . . . 80 Policy, Organization and Discourse in a School District...... 86

111. RESEARCH METHODOLOGY...... 92

Introduction ...... 92 Interpretive Paradigm and Naturalistic Inquiry...... 92 Critical Paradigm and Methodology...... 97 Focus on Conversations...... 102 Person of the Researcher...... 108 Conduct of the Inquiry...... 114 Overview of the Study...... 115 Gaining Access and Building Trust...... 120 Appreciating Context: Document Review and Observations...... 122 Making Sense of the State Testing Policy.... 123 Participant observations...... 123 Interviews...... 126 Document review...... 130 Reflection meetings...... 131 Establishing Trustworthiness...... 132 Member checks ...... 133 Methodological journal...... 136 Peer debriefing...... 137 Researcher involvement...... 137 Balancing experience and reflection 140 Data Analysis Process...... 141 Phase 1: (January to April 1989)...... 142 Phase 2: (May 1989 to March 1990)...... 145 Ethics/Reflexivity/Limits of the Study...... 150

IV. FINDINGS OF THE STUDY...... 159

Introduction...... , ...... 159

Story 1: How Did the State Testing Policy Relate to the Conversations

in the Professional Culture?...... 162 Professional Culture...... 163 Communicational Processes and Conversational Settings...... 170 Educators' Sense-Making and Reactions to the Policy...... 174 Ambivalence...... 176 Professional Dilemma...... 188

ix Impact on Professional Work and Relationships...... 195 Theme 1: Use of the TestResults ...... 200 Theme 2: Effect of Testing on Relationships...... 212 Theme 3: Benefit of Fall vs. Spring Achievement Testing...... 220 Theme 4: Effect of Testing on Local Curriculum...... 223 Theme 5: Effect of Testing on Local Resources...... 241 Summary of Themes 1-5...... 249 Talk about Testing: Discourse Rules...... 250 Rules for Policy and Rules for Culture..252 Articulating a Vision of Education...... 260 Conversations with Constituents...... 261 Summary of Story 1...... 270

Story 2: How Did the Research Process Relate to the Conversations in the Professional Culture?...... 273

Background of Researcher...... 275 Research Process and Conversational Settings...... 278 Discourse Rules in Interpretive Research.... 281 Discourse Rules for Research in Foster...... 284 Effects of Research Methods on Foster Educators...... 291 Evolving Researcher-Researched Relationship...... 298 Summary of Story 2 ...... 304

Story 3: How Might the State Testing Policy and the Research Combine to Bring About New Conversations?...... 305

Theme 6: The Issue of Local and State Accountability...... 309 Research and Collective Conversations...... 327 Important Conditions for Collaborative Discourse...... 335 Summary of Story 3 ...... 339

Summary of Chapter IV...... 340

x V. CONCLUSION 344

Summary of the Study...... 344 Summary of the Findings...... 349 Research Conclusions and Discussion...... 357 Vulnerability of the Professional Culture...... 357 Discourse and Organizational Possibilities..370 Implications of Participatory Research...... 375 Implications for Further Study...... 380

APPENDICES

A. Description of Proposed Research Study...... 383

B. Foster Resources Used for State Mandated Achievement and Ability TeBting...... 385

LIST OF REFERENCES 390 LIST OF TABLES

TABLE PAGE

1. Data Collection Occasions and Activities...... 117

2. Themes 1-5: Anticipated Impact of State Testing Policy on Professional Work and Relationships...... 197

3. Foster Expenditures for Achievement and Ability Testing...... 244

4. Naturalist Research Axioms and Discourse Implications ..... 282

5. State Testing Policy and the Individual and Collective Aspects of Foster...... 337 CHAPTER I

INTRODUCTION

Schools continue to be the focus of public concern and attention as the decade of the 1990s unfolds. Not only are schools expected to insure the intellectual and social well-being of their students, they are expected to insure the economic well-being of the nation through the provision of qualified workers. In the 1980s, when it seemed schools were not living up to these expectations, policy makers, especially at the state level, responded with hundreds of pieces of school reform legislation (Pipho, 1986). How one of these reform strategies, a state testing policy, impacted the work and relationships of educators in the particular context of their school district iB the focus of the present study.

This study examines the implementation of a state testing policy. The examination looks at the implementation process by focusing on the interaction of culture, language, and organization. This interaction shapes how the policy is interpreted and acted upon by the educators in this district. Based on the premise that Bocial reality is

1 socially constructed by human actors (Berger & Luckmann,

1966; Blumer, 1969), and that the particular social

construction of a given group is largely affected by the

interacting forces of its language and culture (Bateson,

1972, 1979; Hall, 1976; Sapir, 1951-cited in Mandelbaum;

Whorf, 1956-cited in Carroll; Wilden, 1987), this study

investigates how a specific set of educators enacted a

state testing policy within the socially constructed

organizational reality of their school district*

This research is primarily concerned with the

discourse of local educators as the means through which

their individual sense-making about the policy is revealed

and their collective response to it is created and

sustained. Recognizing that human actors use language to

construct their social reality and to incorporate a new phenomenon into their organizational reality, the

particular discourse of particular actors is an appropriate

focus for examining the influence of a newly mandated

policy on their actions. State policy makers intend school

reform legislation to improve the education of students.

How a state testing policy is implemented in a school district is determined by how local actors make sense of

it. Its implementation iB therefore Btudied in terms of the conversations of local actors. 3

Background for the Study: Policy Implementation Literature

The literature on educational policy implementation

includes extensive studies of the process of implementing

federal policy at local levels (Bardach, 1977; Berman &

McLaughlin, 1977, 1978; Elmore, 1977; Elmore & McLaughlin,

1982; Pressman & Wildavsky, 1984; Sabatier & Mazmanian,

1980; Van Meter & Van Horn, 1975; Weatherley & Lipsky,

1977). In addition, there are studies of the uses of

different policy tools in different organizational contexts

{McDonnell & Elmore, 1987; Clune, 1986). Furthermore,

increasing attention is being given to the implementation

of state educational policies, focused especially on the

current educational reform movement (Chance, 1988;

Darling-Hammond & Berry, 1988; Ginsberg Jk Berry, 1990;

Johnston £l Niedermeier, 1987;Mitchell & Encarnation, 1984;

Murphy, 1980; Orlich, 1985, 1989; Timar & Kirp, 1988,

1989).

Fewer Btudies, however, have focused on the

implementation of state level policies within the specific

organizational contexts of particular school districts and

from the perspectives of individual school administrators.

Studies which examine policy implementation in light of the

idiosyncratic nature of particular contexts and the world views of local actors are now being called for (McLaughlin,

1987; Miller & Lieberman, 1988; Wimpelberg, Teddlie, &

Stringfield, 1989). McLaughlin (1987) invites a new 4

generation of policy analysis which will integrate the

macro world of policymakers with the micro world of

individual implementors: “The problem for analysts

comprises linking the nominalistic world of the street

level bureaucrat to the systemic patterns that comprise the

world of policymakers" (p. 177).

By focusing on the situations where macro forces meet

micro realities research can shed light on the

organizational and individual aspects of policy

implementation. Timar & Kirp (1989) emphasize that a new

theory of institutional reform would focus on improving the

health and competence of the schools as organizations. This

emphasis on the importance of the particular organizational culture of a school in order to understand the

implementation of a policy or program is echoed by Goodlad

(1988-89):

Because policy makers and many school reformers appear to lack understanding of the school as a culture, they continue to recommend a relatively narrow array of interventions having little or nothing to do with improving the health of the school ecosystem. They continue to assume that schools are like factories. More input in the form of courses, rules and regulations, hours, or even materials will result in greater output. If the output is not judged sufficient, more of the same will do the trick....I am convinced that we will continue to make only peripheral and cosmetic changes in schools until we approach understanding and improving them from a quite different perspective. This perspective begins with viewing the school, not as as factory, but as a culture or ecosystem which will function effectively only if each of the components is healthy and the relationships among these components likewise is healthy, (p. 5) Within the understanding of schools as culturesj there

is a caution not to turn them into reified, concrete entities apart from the people who comprise them and who give them meaning (Blumer, 1969; Foster, 1989; Greenfield,

1980; 1983; Theodossin, 1982). Instead, an examination of

the "subjective constructions whereby meanings are realized

in institutional form" (Foster, 1989, p. 112) is what is needed in the study of educational administration. Using the interpretivist approach, a useful direction for research is "to probe how administrators make sense of their environment, how they attach significance to particular events, and the ways through which

interpretation of reality take place" (Foster, 1989, p.

112). Thus, the implementation of a policy can be studied by examining how local administrators interpret it and make it a part of their organizational context.

The suggestion to study administrators is reinforced by

Anderson (1990) who sees the "meaning managing" role of administrators as crucial to how the members of a school organization construct their social reality. He argues for researchers to take a critical perspective in studying the sense-making processes of administrators in order to uncover how administrators are legitimating certain aspects of the social reality. In Anderson's words:

Accounts are urgently needed that describe how administrators attempt to manage the meaning of their organization and who benefits from the resulting social constructions. (1990, p. 51) 6

In keeping with the call for a new generation of policy analysis, this study examines how the state testing policy is interpreted by educators, especially administrators, situated in the context of their school district.

Statement of the Problem and Research Questions

Stated in general terms, the research question for this study is: How do a set of local educators, in the context of their school district, make sense of and implement a new state testing policy as revealed through their discourse?

The examination focuses on discourse because it is seen as the primary means through which the underlying forces of culture, language, organizational reality, and individual and collective sense making are manifested. Discourse is also the means through which organizational members can change the social reality they have created and maintain.

The investigation addresses the following specific research questions:

1. How does the state testing policy relate to the conversations in the professional culture of local educators in the school district?

a. How do educators make sense of the policy, including their initial reaction to it?

b. How do educators see the policy influencing their work and relationships?

c. What rules for discourse shape the state testing policy?

d. What rules for discourse shape the professional culture of local educators? 7

2. How does the research process relate to the conversations in the professional culture of the school district?

a. What rules for discourse shape the research process in this organization?

b. How doeB the researcher influence the individual conversations about the state testing policy?

c. How is the researcher influenced by conversations with the educators in this organization?

3. How might the state testing policy and the research process combine to bring about new conversations among educators in the school district?

a. How might local educators reframe the way in which the state policy holds them accountable?

b. Can the research process be a catalyst for collective conversations regarding how educators are being held accountable?

c. What organizational conditions facilitate the start of new conversations?

Methodology

Based on the philosophical traditions of phenomenology and hermeneutics, the interpretive social science paradigm is most appropriate for addressing the question of meaning making among human actorB in social groups (Berger &

Luckmann, 1966; Blumer, 1969; Burrell & Morgan, 1979;

Erickson, 1986; Lincoln & Guba, 1985; Morgan, 1983;

Schwartz & Ogilvy, 1979). The scientific concern within the interpretive paradigm is how particular humans socially and symbolically construct their social reality in a given place and given time, recognizing that their culture, language and history influence their current actions and

interactions. Given the purpose of this inquiry, to examine

how the educators in a school district make sense of and

implement a new state policy, a case study focusing on

their interpretations of and actions toward the policy was

conducted. In order to uncover their interpretations

through their conversations I became a participant-observer

in their midst, engaging in conversations and actions with

them. As Heron (1981) contends:

To explain human behavior you have, among other things, to understand [symbolizing] activity, and fully to understand it involves participating in it through overt dialogue and communication with those who are engaging in it (p. 23).

In addition to focusing on the meaning-making of the

social actors, this study attempted to serve as a catalyst

for self-understanding and to aBk the actors why they

interpreted the policy as they did and what alternative

future actions they might take toward the policy. In this

regard, the inquiry also fits the critical social science

paradigm, in which the concern is to critique how it is

that members construct their social reality as they do

(Anderson, 1987, 1989, 1990; Carr & Kemmis, 1986; Deetz &

Kersten, 1983; Fay, 1977; 1987; Lather, 1986; Simon &

Dippo, 1986). 9

Rationale for the Inquiry

Policy makers assume that their efforts at the state

level will influence school personnel to improve their practice for the benefit of students. Studies of what

actually happens when new policies are incorporated into the ongoing organizational lives and individual perspectives of local educators can contribute to the making of effective educational policy (Cohen, 1987; Healy,

1986; McLaughlin, 1987; Wimbelberg, Teddlie, Stringfield,

1989). Healy (1986) argues that interpretive inquiry can contribute to the making of effective policy "through

improving our understanding of ourselves and others, and

thereby opening up new possibilities for social practice"

(p. 390).

Healy's argument for improved, interpretive policy making requires free and uncoerced communication in order

for different actors to bring their multiple realities to

the discourse and develop new shared understandings and

images of one another from which to generate policies. This

emphasis on language, discourse, and dialogue in the policy making process is shared by other researchers seeking to understand how the micro and macro levels of policy analysis might meet (Cohen, 1987; McLaughlin, 1987) and how administrators interpret their work and relationships

(Anderson, 1990; Blumberg &. Greenfield, 1986; Foster, 1989;

Donmoyer, 1985; Pugh, 1987; Taylor, 1986). 10

The focus on "policy and conversation" in this study draws attention to the issue of how communication practices and patterns can either reinforce or provide the opportunity to alter existing social relations (Deetz &

Kersten, 1983; Healy, 1986; Wilden, 1987).

Delimitations of the Inquiry

In a description of any inquiry it is important to describe what the Btudy is not. This case study does not account for macro political, economic, historical and social forces which shape the lives of the members of this school district. In other words, although such issues as who has the power to control what happens in schools, and what past political, economic and Bocial choices led to current school funding practices are important to the current context of the educators in this district, the case study does not directly attend to these issues. Neither does it focus on all the possible conversations in the school district.

Instead the inquiry iB confined to a phenomenological case study of administrators’ (and some teachers’) sense-making during their initial response to and preparation for implementing a newly mandated state testing policy. The conversations of interest are those pertaining to the Btate policy between and among administrators and 11 teachers, that is, among the professional educators within the district. The actors of interest are the central office administrators responsible for curriculum and instruction, the building principals and assistant principals, and those teachers who teach the grade levels and Bubject are&B specified to be tested by the state policy, and teachers involved in curriculum revision (because the state testing mandate could affect deliberations about the content of the local curriculum).

The study only tangentially includes conversations between educators and parents and between educators and board members. It does not follow the political conversations between administrators and board members or between administrators and representatives of the teachers' union. It does not focus on instructional conversations between teachers and students in classrooms. Rather it focuses on the verbal behavior, oral as well as some written communications, which manifests administrators* and teachers* individual and collective understandings of the state testing policy. Nonverbal behaviors are briefly studied in terms of the clueB they provide to the nature of the relationships among the teachers and administrators in the district (Galloway, 1977; 1984; Goffman, 1971; Hall,

1974; 1976; Rosenstock-Huessy, 1970). 12

State Testing Policy

In July 1987 the 117th General Assembly of Ohio passed

Substitute House Bill Number 231 which mandated that each school district give an achievement test and an ability test to all students in grades 4, 6, and 8 beginning no later than the 1989-90 school year

3301.078). Administrative rules accompanying the bill specified that the achievement test be one of ten commercially published norm-referenced tests on a state approved list and that the subject areas of reading, language arts and mathematics be tested. The student achievement scores were to be compared to student "school and mental ability" and together the tests were "intended to show how well students measure up to their abilities"

(Ohio Department of Education, 1987, p. 4; AC Rule

3301-12-03) .

School districts are required to aggregate the test scores and report to the Ohio Department of Education and their own communities (Chapter 3301-11):

a) the number and percent of Btudents who score at or above selected score levels;

b) average scores for the district; and

c) number and percent of students who achieve as well as, higher than, or lower than predicted by ability test performance.

The Ohio Department of Education was charged to compile a report of student achievement for the state as a whole 13 and each district. Although the original legislation specified that school district identities would not be revealed in the state report of the test scores, subsequent

1989 legislation specified that not only would the scores for each district be identified but those from each school building would be published as well (SB 140).

In addition to requiring districts to select and administer achievement and ability tests, the 1987 state policy mandated the development of a statewide "proficiency test" which would be given to all ninth graders for the first time in the fall of 1990

The proficiency test assesses student skills in four subject areas: reading, writing, mathematics and citizenship. It must be passed by students between their ninth grade and the end of their twelfth grade year in order to receive a diploma from their local school district. If students repeatedly fail to pass the ninth grade test, yet complete the local district requirements for graduation, they will receive a "Certificate of

Attendance" rather than a diploma.

In addition, the state is developing a twelfth grade proficiency test. Students who pass the ninth grade proficiency test, but not the twelfth grade one, will receive a "Diploma of Basic Competency". Those who pass the twelfth grade state proficiency test and meet any other criteria established by their local school board will 14 receive a "Diploma with Distinction", and those who pass the twelfth grade test and meet additional criteria established by the State Board of Education will receive a

"Diploma with Commendation" (RC 3313.61).

The authors of the state testing legislation intended for school districts to align their local curricula to address the content and skills assessed in the achievement and proficiency tests. They expected the information from the tests to help teachers and administrators learn how their students were currently performing and what should be done instructionally to improve student performance. Such tests would also provide the public with comparable information from each school district across the Btate (M.

Fox, personal communication, January 30, 1990; E. J. Watts, personal communication, January 21, 1990).

To appreciate how educators received this new piece of legislation it is useful to understand the background of previous state testing policies. In 1983 the Ohio Board of

Education promulgated new minimum standards for elementary and secondary schools which required each district to prepare and submit local "courses of study" for each subject taught (essentially the written curricula of the school district). Furthermore, for the subject areas of

English composition (writing), mathematics and reading, districts were obliged to locally develop competency based education in which performance objectives were to be 15 established for students to accomplish, and students were to be periodically assessed on these objectives. Districts were to begin the competency based testing by the 1984-85 school year with full implementation completed no later than the 1989-90 school year.

In essence the 1983 policy directive from the state board of education required school districts to periodically test student achievement based on their local curriculum objectives, and to have this process fully in place by the 1989-90 Bchool year. During 1987, however, the

Ohio legislature passed legislation which mandated districts to test student achievement in grades 4, 6, and 8 using one of ten state approved tests by the 1989-90 school year, and to prepare students to pass a state developed proficiency test for grades 9 and 12 (to be administered for the first time in the 1990-91 school year). Thus, between 1983 and 1987, the emphasis in state policy shifted dramatically from a local to a state definition of what content and shills students should be tested oh. The competency tests were developed or purchased based on their fit with a locally developed curriculum; the achievement tests were approved by the state and proficiency tests were designed by the state. An additional feature of this combination of contrasting directives is that the two sets of policies (1983 local focus and 1987 state focus) have roughly the same effective implementation dateB. 16

The 1987 legislation did allow some flexibility in the choice of norm-referenced achievement tests. Rather than mandating that all districts give the same achievement test, the 1987 policy allowed districts to select the test best Buited to the local curriculum from a state approved list of ten commercially prepared tests. In addition, in order to comply with the competency requirement of the 1983 minimum standards, many districts selected a commercially prepared norm-referenced achievement test to use as their periodic assessment of pupil performance objectives rather than undergo the time consuming and costly effort of developing a criterion referenced test to match their local course of study.

For some districts, including the one which is the focus of this study, the use of a commercially prepared norm referenced achievement test as a means to assess pupil performance on local objectives was seen as an expediency measure to respond to the 1984 requirement of the minimum standards. The expectation was that by the time the 1989-90 full implementation date arrived they would have revised their courses of study and developed appropriate competency measures to assess student performance on the locally developed objectives. The 1987 testing legislation in effect "stopped them in midstream" of implementing the 1983 policy. Rather than continue their efforts to find or develop assessments which were aligned with their local 17 course of study, they began to consider how their students would fare on one of the state selected achievement tests and the state designed proficiency test.

The school district Btudied in thiB inquiry, pseudonatned "Foster", was one of the districts hoping to revise its course of study and competency testing practices in the areas of language arts and reading, as well as mathematics, when the 1987 legislation shifted to a statewide assessment of these subject areas. Foster educators had planned to move away from their use of the

California Achievement Test (CAT) as their competency measure. With the passage of the 1987 policy, however, they continued to use the CAT but administered it in different grade levels to comply with the state requirement to test grades 4, 6, and 8. In short, Foster had been using the

CAT, if reluctantly, to comply with the 1983 competency mandate, but was anticipating replacing it with a more appropriate assessment better suited to the revisions they were making in their courses of study. The 1987 state policy shifted their attention to state rather than locally developed assessments.

At the beginning of this study in January 1989, the school district was ten months away from the requirement to test fourth graders by the fall of 1989, and sixth and eighth graders by fall of 1989 or spring of 1990, whichever they chose. That is, the state policy required them to give 18

the fourth grade test in the fall, but allowed districts

the option of giving the sixth and eighth grade test in the

fall or the spring.

As described in more detail in Chapter IV, Foster

educators made the decision in the fall of 1988 to invest

in computer hardware and software to do their own scoring,

analysis, and reporting of the achievement and ability test

data. Rather than sending the tests back to the publisher

for scoring, they decided to develop the internal capacity

to score, analyze, and report the data, thereby gaining

some control over the timeliness of the scoring, the

variety of analyses available, and the formatting of the

reports for different audiences (parents, teachers,

administrators, and school board). Consequently, Foster

implemented the achievement/ability portion of the 1987

state testing policy one school year ahead of the mandate.

(In the summer of 1989 the state legislature rescinded the

requirement to test fourth graders in the fall, and allowed

districts to choose fall or spring testing for this grade.

Foster subsequently opted to test all required grades in

the spring of each year.)

Given the shift of emphasis from locally determined to state determined tests, the question of whether Foster educators would align their local written curriculum and

instructional practices with the content, skills, and instructional practices implied in the new state 19

achievement and proficiency tests became an important issue

for this policy implementation study. Prior to considering

the effect of the state policy, however, some background is

offered on the community and schools of Foster.

Community and Schools of Foster

As part of a Midwestern metropolitan area, Foster has

recently undergone a dramatic transformation in size and

character. In the last decade it has emerged from a small

farming community to a rapidly growing city. Its new

residents are typically professionals, business executives

and middle managers who work in the nearby metropolis or in

the many new businesses expanding into Foster and the

surrounding areas. Large, expensive new homes are being built in combination with more modest dwellings and rental units. The school population is one of the fastest growing

in the state, increasing by over 200% in the last decade

and anticipating doubling its size again in the next five years. Of the ten schools open when the study began, six have been built in the last five years. Approximately 600 new Btudents were enrolling throughout each school year.

With this expansion Foster has been able to hire new teachers and administrators, selecting what they consider the best from Beveral thousand applicants.

Currently, some Foster residents are expressing concern over the increase in taxes accompanying thiB rapid growth, 20 and the school district has been criticized for what some consider to be fancier-than-necessary new buildings and programs. A continuing operating levy and two bond issues which would have built two additional elementary Bchools and a natatorium were defeated in November 1988. An emergency, one time operating levy was subsequently passed in May 1989, thereby avoiding the laying off of 106 teachers new to the district that year.

When this study began in January 1989 the district organization included seven elementary, two middle, and one high school serving approximately 7000 students. The central office included a superintendent, a treasurer, two assistant superintendents (one for operations and development, the other for curriculum and instruction), four directors and seven coordinators. Each building had a principal, each middle schodl had an assistant principal, and the high school had two assistant principals and a dean of students.

In addition to the focus on central office administrators responsible for curriculum and instruction, particular attention was given in this study to the educators within the two middle schools. Students in grades

6, 7, and 8 attend the Foster middle schools. Because the state achievement testing is required for grades 6 and 8, and because the proficiency test will be administered in the fall of the ninth grade year, the curriculum and 21 instruction of the middle schools could be most affected by the 1987 state testing policy.

In addition, achievement testing is required for fourth graders in the elementary buildings, and high school students will take the ninth and twelfth grade proficiency tests. Consequently, elementary and high school principals were included in the study, and the deliberations of teachers in high school department meetings as well as district wide curriculum revision meetings were observed.

Definitions

In order to clarify how some specific words and clusters of words are used in this study, the list below delineates their meanings in accordance with the definitions given them by various scholars. In some cases these definitions are elaborated upon in Chapter II. action:

Refers to a physical act with meaning (as opposed to behavior which can refer to just a physical act). That is, an "action" has meaning attached to it (e.g., see Erickson, 1986, p. 126). language and discourse:

Language (in the broadest sense) is "any structural system of codifiable signB by means of which a particular group of people communicate meaning and regulate their activities. Language involves a set of signs (vocabulary) and relational ruleB (grammar) as well as a means of discourse in these signs (communication system)" (Evered, 1983, 126, and 129). 22

Wilden (1987) distinguishes between language and discourse by noting that discourse, unlike language, always has a subject (some person talking) and a subject matter (something about which the person is talking to another person) (p. 132).

In this study I use the terms ‘discourse’ and •conversation* interchangeably. Essentially I am focusing on the ongoing talk of educators as they deliberate about the state testing policy. code, communication, text and context:

A code is a system of constraints, a code of rules, that permits communicants... to construct mutually understandable messages (Wilden, 1987, p. 319).

Communication, literally, "to share or to make common", or "the sending and receiving of messages" (Wilden, 1987, p. 70)

Text is the content of the message in a communication (Wilden, 1987). It can refer to "a particular concrete manifestation of practices organized within a discourse" (Lewis & Simon, 1986, p. 2)

Context includes "all the symbolic, imaginary and real relations of daily life" (Wilden, 1987, p. 70). In this study, the term context refers to the particular subculture of the school district in which educators live out their work lives. When referring to context I am referring to all the relationships included within the organizational subculture of the educators within this school district. In addition, the term acknowledges that the context is continually created and recreated by the actions of the people in the different settings.

To further clarify, to communicate (share meaning) one must use and understand the code. In communicating, one is simultaneously conveying text (the content of the message) and context (the underlying relationships which give meaning to the text). Any communication takes place within the rules of a particular discourse, within the language structure of a cultural group. culture and organizational culture:

Culture is "the transmitted and created content and patterns of valueB, ideas, and other symbolic - meaningful systems as factors in the shaping of human behavior" (Kroeber & Parsons, 1958, pp. 86-87), 23

or: "learned and shared standards for perceiving, believing, acting and evaluating the actions of others" {Goodenough, 1981, p. 62)

Organizational culture is "the pattern of basic assumptions that a given group has invented, discovered, or developed in learning to cope with its problems of external adaptation and internal integration, and that have worked well enough to be considered valid, and therefore to be taught to new members as the correct way to perceive, think and feel in relation to those problems" (Schien, 1984, quoted in Blumberg & Greenfield, 1986, p. 224). norms and roles:

Norm refers to "that kind of guide for action that is supported by social sanctions, negative ones providing penalties for infractions, positive ones providing rewards for exemplary compliance" (Goffman, 1974, p. 95) .

Role refers to "those expectations of one’s behaviors by significant others in the relevant social structure" {Wirt & Christovich, 1988, p. 26).

or: typifications of particular sets of actions, ascribed to and passed on as knowledge to particular actors in a society or organization (Berger & Luckmann, 1969, p. 73). policy:

In general, a plan of action which describes what people should do under given conditions to bring about some desired state (Kerr, 1976). professionalism:

Members sharing a common body of knowledge and using shared standards of practice in exercising their knowledge on behalf of clients (Darling-Hammond & Berry, 1988, p. v) . professional culture:

In this study, the term refers to the subculture created and sustained by professional educators in Foster, specifically central office administrators, building administrators, and teachers. 24 symbol;

"A symbol is a sign which denotes something greater than itself, and which calls forth the association of conscious or unconscious ideas, in order for it to be endowed with full meaning and significance... All symbols are created subjectively and are invested with a particular kind of subjective meaning" (Morgan, Frost & Pondy, 1983, pp. 4-5).

Overview of Chapters Two through Five

The particular setting for this study was the professional context of administrators and teachers in the

Foster school district. The study focused on their discourse, their conversations about the state testing policy, in an attempt to uncover its meaning in their individual lives and their collective implementation of the policy.

Chapter II describes the theoretical, philosophical and research background for studying the meaning-making of social actors in educational settings. It provides an overview of the literature on policy implementation, with particular attention to educational reform policies and testing policies. Chapter III describes the research methodology including a rationale for the caBe study approach within the interpretive and critical research paradigms. Chapter IV presents the findings of the study in terms of three distinct, yet related, Btories. The stories pertain to how educators in their professional subculture made sense of the policy as manifested by their discourse; 25 how the research process influenced and was influenced by their conversations; and how the state policy and the research process might combine to create a new set of conversations about the way in which educators are held accountable. Chapter V presents research conclusions, discussion of related ideas, and implications for further study. CHAPTER II

THEORETICAL, PHILOSOPHICAL AND RESEARCH BACKGROUND

Introduction

In this chapter the theoretical, philosophical and research frameworks for the study are presented. The central focus is on how language, organization, and policy interact to shape the conversations among administrators and teachers in a school district. This is not a comprehensive review of all of the relevant literature in the fields of organizational theory, policy sciences and sociolinguistics. Instead, illustrative examples from the literature are presented to connect my understanding of the nexus of language, culture, organization, policy, and research to the present study.

The chapter starts with a focus on language and organizations and the ways in which they are shaped by and shape the culture of the collective. It then proceeds to a more detailed look at the nature of discourse within a specific segment of an organization. The nature and role of policy and policy analysis is then explored, followed by a

26 27

review of the literature on the implementation of public

policy, particularly those policies associated with the

educational reform movement. A description of the issues

surrounding state testing policies is presented next.

Chapter II concludes with a discussion of how the

elements of policy, organization and language combine to

shape the conversations of members of a specific

organizational context. This integration is offered as an

emergent form of policy analysis, one which both focuses on

the particular contexts in which practitioners work and

takes into account how practitioners make sense of and create their response to external policy initiatives in light of that context.

Although they have distinct academic homes, the topics of language and organizational theory are inextricably linked with each other and with the notion of culture. In an explanation of human action, it is difficult to refer to one element without drawing in the other two, because language, culture and organization interact, constrain, and shape each other. That is, it is difficult to discuss language without referring to its cultural base; or to describe an organization without referring to its creation and maintenance through language; or to focus on 28

organizational culture without describing how humans create their shared meanings through symbols, most prominently, the symbols or codes of language.

There are, however, different streams of academic research on language and on organizations, so I will touch on each of them separately without denying they are intellectually intertwined. In this discussion, the notion of culture is used in two ways: it is linked with language in terms of its most general sense of the shared beliefb and patterns of living within broad racial, religious or regional groups, and it is linked to organizations in terms of the specific patterns of acting and the social ethos established by smaller groups of people who make up segments or subcultures of large organizations and institutions.

Language and Culture

Fundamental notions about language and culture are provided in the works of Edward Hall, Edward Sapir, and

Benjamin Lee Whorf. Hall (1976) maintained that each culture has ways of looking at life which are so unconscious we as humans do not realize that we created them in the first place or the ones belonging to our culture are not the only ones among human beings. These ways of looking include perceptions about time, space, selfhood, interpersonal relations, norms for behavior, 29 roles, and responsibilities. That is, the ways of thinking, knowing, and behaving are different in each culture, and are affected by its language.

Sapir (cited in Mandelbaum, 1956) described the prevailing influence of cultural patterns on human actions:

If we can allow that normal human beings, both in confessedly social behavior, and often in supposedly individual behavior, are reacting in accordance with deep seated cultural patterns, and if, further, we can show that these patterns are not so much known as felt, not so much capable of conscious description as naive practice, then we have a right to speak of the "unconscious patterning of behavior in society", (p. 548).

Furthermore, Sapir argued these cultural patterns are influenced and constrained by the language of the community which affects how its members perceive "reality". In his words:

Human beings do not live in the objective world alone, nor alone in the world of social activity as ordinarily understood, but are very much at the mercy of the particular language which has become the medium of expression for their society. It is quite an illusion to imagine that one adjusts to reality essentially without the UBe of language and that language is merely an incidental means of solving specific problems of communication or reflection. The fact of the matter is that the "real world" 1b to a large extent unconsciously built up on the language habits of the group.... We see and hear and otherwise experience very largely as we do because the language habits of our community predispose certain choices of interpretation, (cited in Carroll, 1956, p. 134) 30

The assertion that the language of a society

predisposes its members to adopt certain interpretations of

"reality" was echoed by Whorf (cited in Carroll, 1956):

And every language is a vast pattern-syBtem, different from others, in which are culturally ordained the forms and categories by which the personality not only communicates, but also analyzes nature, notices or neglects types of relationships and phenomena, channels his reasoning, and builds the house of hiB consciousness, (p. 252)

The subtle, unconscious aspect of this cultural

channeling of what is noticed, what is ignored, what is known, and what is considered worthy of knowing is an

important aspect of the power of language to both inhibit and create the possibility of change within a society. If human beings do not consciously realize the cultural patterns that are constraining their thoughts and action, the possibility of changing their thoughts and action is severely limited. The first step in bringing about a change

is making the existing patterns available to one's "house of consciousness" and then overtly developing a new language pattern to address the change in thought and action. This idea will be developed more fully in a later section but its rationale relies on an appreciation of the subtle, yet powerful, influence of language in shaping the thoughts and actions of a cultural group. 31

The ideas of Sapir and Whorf have become the basis for

the field of sociolinguistics in which the complex

interrelationships of language and society are studied. In essence, sociolinguists study the co-variations of social and linguistic phenomena. A shorthand way of referring to

the influence of language on society is known as the

'Sapir-Whorf’ hypothesis. As summarized by Trudgill <1983):

The hypothesis is approximately that a speaker’s native language sets up a series of categories which act as a kind of grid through which he perceives the world, and which constrain the way in which he categorizes and conceptualizes different phenomena. A language can affect a society by influencing or even controlling the world-view of its speakers, (p. 24)

Social Construction of Reality

Sociologists Herbert Blumer (1969) and Peter Berger and

Thomas Luckmann (1966) provided a theoretical description of how humans create the language and culture which shape their own lives. The underlying premise of their theory iB that reality is socially constructed through the actions and interactions of human beings. Referred to as 'symbolic interactionism’ and built upon the work of George Herbert

Mead, Blumer’s explanation begins with a focus on human action as the basis for human culture. That is, culture is derived from what people do, and the social structure of a culture refers to the relationships established as people act toward one another. In this way, the life of a society 32

is an ongoing process of "fitting together the activities

of its members" {Blumer, 1969, p. 7).

Human action is based on the meanings people give to the 'objects' of their actions. These meanings are derived

from symbolic interaction; that is, the meanings are not

intrinsic to the objects but grow out of how each person,

in combination with the others around her, defines the object. In Blumer's termB the 'objects* can be physical

(e.g., bicycles, trees); social (e.g., mother, friend, student); or abstract (e.g., justice, compassion, principles). But the nature of the object depends upon the meaning it holds for the person who attends to it: "This meaning sets the way in which he sees the object, the way

in which he is prepared to act toward it, and the way in which he is ready to talk about it" (p. 11).

The individual’s meaning is affected by the way in which the people surrounding him or her see the object, and thus members of groups come to see and define objects in similar ways, and these common meanings comprise the basis of their created culture. Choosing to focus upon, and to define and interpret certain objects in the same way provides the basis for different individuals to forge their actions together and to interact in stable, routine, or habituated ways. The repetition of stable instances of interactions creates organizations and societies. The background or history of past joint action provides the 33 context for interpreting current actions* This process does not happen by itself, however. It relieB upon the constant meaning-making of the individuals who are members of the group or culture. Different groups will choose to attend to different objects, thereby creating their distinctive social realities. That is, people individually and collectively create and sustain their social realities by continually interpreting the physical, social and abstract objects t.o which they choose to attend.

Berger and Luckmann (1966) explain the social construction of reality in a similar way. Also influenced by the ideas of George Herbert Mead, as well as those of

Max Weber, Emile Durkheim and Albert Shutz, they focus on how social reality 1b internalized by individuals. To

Berger and Luckmann, society is comprised of two features:

"objective facticity and subjective meaning" and the key question is how is it that subjective meanings become objective facticities; or "how is it possible that human activity should produce a world of things?" (1966, p. 18).

Berger and Luckmann define reality as "a quality appertaining to phenomena that we recognize as having a being independent of our own volition" (p. 1). That is, although reality is socially constructed, it cannot simply be "wished away". They argue that everday reality is subjectively interpreted by human beings; it is taken for granted; and it originates in the thoughts and actions of people and is maintained as real through these ongoing thoughts and actions. That is, the Bubiective meanings of people in proximity to each other are objectified and in this way the group constructs their own inter-BUbjeotive commonsense world. The objects people individually and collectively choose to give meaning to become the

"objective" reality of their everyday lives. Such everyday reality appears to the individual as objects which have an order or pattern to them before the individual sees them; that is, they appear as if they exist outside of the meaning one gives to them.

Language provides the vehicle for this objectification of subjective meanings:

The language used in everyday life continuously provides me with the necessary objectifications and posits the order within which these make sense and within which everyday life has meaning for me. ...Language marks the coordinates of my life in society and fills that life with meaningful objects. (1966, p. 22)

Through language one can share everyday life with others.

In fact, everyday life is primarily life with others via a shared language.

Language 1b a special case of the process of objectification since it is the human production of signs which are explicit expressions of subjective meanings.

Language (defined by Berger and Luckmann as a system of vocal signs) is the way humans maintain their common 35

objectifications of everyday life. It has the quality of an

objective facticity: that is, it is external to myself and

I must follow its rules; yet, at the same time, it allows

me to objectify my subjective experience and thereby share,

or make common, my experience with others. ThiB sharing of

experiences is most often realized in face-to-face

interactions, where I become aware of the subjectivity of

the other. These interactions, although flexible, become

patterned, or routinized over time so that I come to know

what to expect of the other in the interaction.

In addition to allowing individuals to reveal their

current subjective experiences, language allows the common meanings of a group to be detached from the here and now

and provides them with an "objective repository of vast

accumulations of meaning and experience, which it can preserve in time and transmit to following generations"

(1966, p. 37). Thus new generations are given a "social stock of knowledge" (p. 41} which is received as if the knowledge were "objective" and not inter-subjectively constructed by previous human beings.

Language passes on these objectifications and over time interaction patterns and social knowledge become

"institutionalized". That iB, the members of the group produce a social order in which their activities become habituated, routinized, stable. There is Ibsb choice regarding how to engage in interactions with another; 36

individuals are freed from moment-to-moment decisions regarding how to predict the actions of others and how to act themselves. In Blumer's terms, they know how to fit their actions together. The institutionalization process provides stability to a society; a stability which supports or gives the opportunity for occasional change. That 1b , freed of the decisions of how to act in each and every instance of the day, the members of a society or organization can decide to reconsider a particular action in order to change it.

Thus the habituations and objectifications which created the institution are seen and felt as an external reality, beyond, outside of, over and above, the human beings who constructed it. The institution has a history and a future independent of the individual, and it is apprehended as an objective facticity, a facticity which impinges upon the individual, is felt as real, even if not completely understood by him or her. In feeling the reality of the institution, an objectivity which cannot be wished away, individuals can easily forget that it has been constructed by them. It remains a humanly produced reality, dependent on their reproduction of it in their daily actions. Berger and Luckmann note the paradox inherent to this construction: "Man is capable of producing a world that he then experiences as something other than a human product" (1966, p. 61). 37

Previous human actions and interactions are powerful

influences, nonetheless, on current social reality. For

example, the historic human choices represented in capitalism, socialism, democracy, totalitarianism, racism,

sexism, and the exploitation of natural and human resources have definite consequences for the options available to

people living today. As individuals and as collectives, humans have the ongoing choice of reinforcing or changing

these inherited decisions.

Discourse

As noted earlier, the production of the human world iB dependent upon and facilitated by language. Language, however, is a broad term defined as "any structured system of codifiable symbols by means of which a particular group of people communicate meaning and regulate their activities" (Evered, 1983, p. 126). As such, the term language is used to refer to the set of signB available to members of a group when they choose to talk to'each other about a given subject. The particular content of their talk, the actual words they select to use, and the rules which govern their specific conversations are more appropriately referred to by the term ‘discourse’.

As distinguished by communication theorist Anthony

Wilden (1987), discourse, unlike language, always has a subject and a subject matter: "Discourse is some people 38

talking to some other people about some relationship or other" (p. 132).

Although the specific discourses of interest to this

study will be described later in this chapter, it is important to address Wilden*s contention that "in any

society, we can expect to find dominant and subordinate discourses" {1986, p. 132). That is, access to how language shapes a given society or culture is not necessarily equally distributed to all members. Some people have greater influence over just how the language of a culture will Bhape the construction of reality. Drawing on the work of Gregory Bateson (1972), Wilden maintained that all human behavior is a form of communication, and further asserted:

The dominant forms of communication in a given society (both verbal and nonverbal) define and constrain itB ways of seeing and believing, and knowing and judging. They form the ground of what the dominant members of the society accept as true and false, legal and illegal, legitimate and illegitimate, (p. 132)

The most universal example of dominant-Bubordinate communication relations, Wilden argued, is the dominance of male discourse over female discourse. In her review of the literature on the relationship of gender to the subject matter and patterns of talk in mixed male-female group settings, Edelsky (1978) described how this dominance is displayed. She concluded "the less powerful have restricted rights to talk" {p. 8). Being interrupted more often and 39 having the topics they introduce ignored in the conversation are two of the ways in which women are governed by the rules of the dominant discourse. In their article on the silencing of women in a university graduate course, Lewis and Simon (1986) described how discourse serves as a "mode of governance" which limits the range of possible practices. Drawing from the work of Michael

Foucault, they saw the discourse rules of the classroom structuring the possible field of human action for the students within the educational context. Discourse has the power to limit who students were, what they were able to do, and who they might become.

Culture, Context and Discourse Rules

In this discussion, the use of the terms 'culture* and

'context’ are interrelated. As noted above, by culture I am referring to both the customary beliefs, values and social patterns of broad ethnic and regional groups as well as to the specific patterns of acting of subsets of members in an organization. If effect, the culture of an organization is constantly created and maintained by the actions and interactions of its members. This ongoing, unfolding, and flexible aspect of culture is what I am referring to when using the term 'context*. That is, members of a cultural group in an organization constantly create the context in which they work. Although this context reflects the values 40 and patterns of acting in their culture, the context can change. Members can choose to engage in a different pattern of action, or a different way of interacting. In so doing they are changing their context, and eventually, its underlying culture.

The power of a particular context and its inherent discourse rules to govern human action in a group has been addressed by Bateson (1972), Goffman, (1974, 1983), Hall

(1976) and Wilden (1987). Keferred to as "frames" by both

Bateson and Goffman, the discourse rules embedded in any given context provide a metamessage regarding the social relations underlying the signal being sent on the surface:

A frame is metacommunitive. Any message, which either explicitly or implicitly defines a frame, inso facto gives the receiver instructions or aids in his attempt to understand the messages included within the frame (Bateson, 1972, p. 188).

[Behind the face-to-face interactions], there were large, sturdy, durable institutional forces that distributed the resources of interaction (e.g., power, prestige, social skills) unequally and these forces have the ultimate authority to enforce the framing rules that limit the moves that the interactionist might make (Goffman, 1974, p. xv).

Goffman (1983) maintained that face-to-face contact, which he refered to as the 'interaction order’ of human activity, are governed by "enabling conventions, in the sense of ground rules for a game" (p. 5). Whether applied to the interactions of large groups in society, or to 41 the Interactions among individuals in an organization, these discourse rules extract a great price from some disadvantaged categories of persons. Although usually accepted without question, the rules benefit some members of the society or the organization more than others. In order to participate in the interaction order at all, some members pay a much higher price than others because their participation reinforces the existing dominant-subordinate social relations.

Hall (1976) suggested culture provides a "screen" to the external world which tells us what to pay attention to and what to ignore. Embedded in this screen are rules for the social game of constructing meaning within the particular contexts we create in our everday lives. Wilden

(1987) played with this notion of discourse rules in the title of his book on communication context theory: The

Rules are No Game. Indeed, the discourse rules are not a game but very serious constraints on who we are and who we can become within a given organization or given society.

This study addressed how the culture and discourse rules established in a school district organization were affected by the implementation of a state testing policy.

The issue becames how much the external policy would affect the context educators had created in their organization. 42

Organization and Culture

In this presentation* organizations are considered as subcultures of society, settings where groups of people work together, in businesses, hospitals, or school districts, for example. Organizational theory has developed as an academic field to understand and describe the human and social phenomenon known as the organization. W. Richard

Scott (1981) characterized the history of organizational theory as moving from a view of organizations as closed systems, impervious to environmental influences (1900 -

1960), to seeing them as open systems, highly interdependent with their environments (I960- present).

Within each of these two broad time periods, theorists adopted either a "rational" or "natural" model to explain types of organizations.

The rational systems model emphasizes the belief that organizations are intentionally designed to accomplish specific and explicit goals. In keeping with the focus on accomplishing clear goals, the rational thing to do is to formally and explicitly create rules to govern roles and procedures so members can accomplish the organization's announced purposes. The natural systems model, on the other hand, maintains members lack consensus on specific goals and share only an interest in the survival of the organization. This model emphasizes the unplanned and spontaneous processes of the organization, whose members 43

organically form coalitions and informal structures which

"supplement or subdue mechanically designed rational

frameworks as the basis for organizational behaviors and

beliefs" (Scott, 1981, p. 408).

Another approach to analyzing organizations focuses on

how people make sense of them as socially constructed

arrangements for their work lives. Gareth Morgan (1986)

depicted organizations in terms of the images or metaphors

used to interpret their "reality". In keeping with the

belief that reality is socially constructed through the

meaning we attribute to 'objects' around us, Morgan noted

we bring taken-for-granted images to our analysis of the

social object known as the organization:

Our theories and explanations of organizational life are based on metaphors that lead us to see and understand organizations in distinct and yet partial ways. ...The use of metaphor implies a way of seeing and a way of thinking that pervade how we understand our world generally. (1986, p. 12) (emphasis in original)

Morgan described the implications for seeing organizations as machines, organisms, brains, cultures, political systems, psychic prisons, flux and transformation, and as instruments of domination. Most pertinent to thiB discussion is the metaphor of organizations as cultures. In his analysis, Morgan noted organizations are affected by the larger national culture

in which they develop, and that subcultures within 44

organizations serve to bind members together. Through

shared norms, roles, symbols, rituals, beliefs and

routines, organizational members actively 'enact' their organizational world (Weick, 1979).

In her description of organizations as "oulture-bearing milieux", Meryl Reis Louis (1983) viewed them as:

...distinctive social units possessed of a set of common understandings for organizing actions (e.g., what we are doing together in this particular group, appropriate ways of doing in and among members of the group..."thou shalt" [and] "thou shalt not"), (pp. 39, 43)

The development of these social norms takes place at the inter-subjective level, that is, between and among the subjective meaning-making of individuals. As a group, organizational members intersubjectively negotiate the meaning of the organization (Bormann, 1983; Deetz &

Kersten, 1983). The meaning is always emergent and members are constantly enacting their organizational reality.

Organizations as Relationships

Another way of seeing organizations is to consider them collections of people in various personal and collective relationships, Blumer (1969) views organizations as people fitting their actions together and thereby creating the social reality of the workplace: 45

Symbolic interactionism sees these large scale organizations or molar units...as arrangements of people who are interlinked in their respective actions. The organization and interdependency is between such actions of people stationed at different points. At any one point the participants are confronted by the organized activities of other people into which they have to fit their own actB. The concatenation of such actions taking place at the different points constitute the organization of a given molar unit or large-scale area. (Blumer, 1969, p. 58)

To help make sense out of the "concatenation of such actions" individuals in organizations are given prescribed roles and responsibilities. Roles can be seen as typifications of particular actions ascribed to particular actors within the institution and passed on as knowledge to succeeding generations (Berger and Luckmann, 1966). These typifications are the result of personal interactions in which people repeatedly interpret the actions of each other in patterned and consistent ways. Such repeated interpretations allow individuals in organizations to anticipate the behavior of otherB and thereby fit their actions together. In effect, roles make the socially defined reality of others available to the individual. They also provide stability to the organization by allowing people to get on with the work to be done instead of having to stop every moment and interpret each action of the other as if it were a unique and random event.

Although roles provide a rough description of the actions expected of individuals in an organization, people enact how they will play out their roles through the personal relationships they establish in the organization.

Through the development of personal relationships the actions of the organizations are affected, often in different ways than the roles and formal policies publicly describe. Witness the different picture that would emerge if an institution's formal organizational chart (describing the responsibility and power of individuals based on their roles) were juxtaposed with a chart depicting members according to the "informal" organization (baBed on the influence certain individuals have through their personal relationships).

The social identity provided through the role is fleshed out through the personal identity of the individual

(Goffman, 1971). How much of that personal identity is made available to the other is determined by the personal relationship established between two people. The relationships are developed and defined through interactions. That is, in face-to-face encounters, persons reveal themselves to each other. During an interaction, each person responds to the image he or she holds of the other. The image is partially based on role, but is also based on any personal information about the other that has been received through previous encounters, from observations of the other in different settings, or from the opinions of third parties. In addition to the image of 47

the otheri each person has an image of the relationship

between them. Is it cold, distant, strictly business, that

is, strictly based on role? Or is it warm, friendly, open

to new information about the other? How much a person gives

access to his/her subjectivity depends in part on the

current perception or definition of the relationship with

the other person (Goffman, 1971).

Therefore, in an encounter within an organization each

person has some decisions to make regarding how to act

toward the other. Although they receive a prescription of

how to act through the received role, individuals make

choices regarding whether to stay within that role,

restrict it, expand it, or modify it in some way. Their

choices define what the relationship will be and each

interaction offers the opportunity to reinforce or change

that definition. In this way the social reality of the

organization is created and re-created through the

relationships members establish with one another.

Organizational Policies and Communication

Most organizations promulgate policies which describe how its members shall fit their actions together* Policies describe what a person should do under given conditions* In

effect, they tell members what action to take when specific conditions occur in the organization (Kerr, 1976). In addition to telling members what actions to take when, the 48 policies express the underlying culture and created context of the organization. As noted earlier, this context frames, shapes, and constrains the meaning people make of their actions and interactions. The context proscribes the pattern of social relations that are expected in the institution (Goffman, 1974; Wilden, 1987).

These expectations are expressed through policies. The patterns for behaving, the 'modus operand!’, the way we do things around here, the 'standard operating procedures’ are all ways of telling individuals how they should interact with one another. The dominant context is expressed through policies and these policies influence the way any two people interact.

Furthermore, in the communication taking place during the interaction, messages are conveyed on at least two levels (Wilden, 1987). There is the surface level of the

'text', which is the observable or ostensible content of the message or Bignal. And there is the deeper level of the

'context’, in which the discourse rules or constraints simultaneously describe the personal relationship, specify what can and cannot be Baid, and identify the power relations between the two interacting parties.

Yet within the interaction there is always the possibility that the actors can alter the relationship.

They can decide to shift out of the dominant context, to

Bay the unexpected, to break the rules of the discourse 49

game, and thereby influence the dominant context by

pressing back on it with a new definition of how the

relationship should be established, and how the interaction

should go. Such breaking of the rules, if repeated over

time and in more and more relationships in the

organization, can coalesce to alter the dominant context.

These new forms of interacting can affect how roles are defined and affect the policies which identify what the nature of the relationships among the members of the organization should be.

The creation and re-creation of the organization is done through communication. In each of these face-to-face encounters, organizational members are communicating. Sinoe all behavior is communication, a human organism cannot not communicate (Bateson, 1972; Wilden, 1987). Even silence is a form of communication (Wilden, 1987)..

Summarizing the role of communication as a symbolic activity in the development of organizations, Morgan,

Frost, and Pondy (1983) maintained:

Members of an organization are able to use language, can exhibit insight, produce and interpret metaphors, are able to vest meaning in events, behavior, and objects, seek meaning in their liveB - in short, can act symbolically. This symbolic capacity 1b enhanced by their association in formal organizations bo that institutions develop a history, a common point of view, and a need to process such complexity through symbolic means. Organizations are by their very nature symbolic entities, and a fully adequate theory of them must perforce also be symbolic in content. (p. 4) 50

The most prominent symbolic activity is that of language. In his study of the language of organizational members, Evered (1983) asserts simply: "The language they use defines their reality" (p. 126). He goes on to describe that without the linguistic facility of the members, there would be no organization:

The "organization" has no objective reality (in a positivistic sense), but rather is created daily by the linguistic enactment of itB members in the course of everyday communications between each other; that is, by the way in which members talk, hold discourse, share meanings, (pp. 126-127) (emphasis in original)

Organizations represent a way of creating arrangements of people to accomplish some form of work. Organizations are characterized by "structures" whereby member roles, responsibilities and relationships are defined. Thayer

(1988) describes the centrality of communication in this creative process by declaring that "We say the structures and arrangements of our world into existence" (p. x).

In addition to initially creating organizational reality, members must communicate constantly to sustain the organization. Mumby (1988) points out organizational members are continually confronted with new information from the environment which challenges their ongoing process of organizing themselves. Developing a consensus for how they shall respond to the ambiguous information coming from the environment is a part of the enactment of 51

organizational reality:

A taken-for-granted social reality is created in an organization through the continuous movement back and forth between sense and non-sense. In other words, social actors must frame ambiguous information in terms of what they already know "makes sense". This ambiguous information, in turn, subtly changes an actor’s perception and definition of "organizational reality", (p. 10)

In the present study, the state testing policy represents an example of information coming from the environment challenging the members' understanding of their organization as having a professional culture; that is, characterized by teacherB and administrators, not state legislators, making curricular and instructional decisions.

How these local educators "make sense" of the state testing policy, and how it may change their perception and definition of their organizational reality is the focus of this inquiry.

Summary and Example from Schools

The perspectives presented thus far maintain organizations are social realities constructed and sustained by the communication practices of their members.

The underlying culture of the organization and the ongoing context created by members shape the relationships among members and the rules for their discourse. These relational and discourse rules are expressed in the organization's 52

formal policies, which specify how members should fit their

actions together. The following example of a school

organization illustrates how the particular context,

discourse rules and policies all interact to create and

re-create organizational reality.

School policies describe the roles and behaviors for

teachers, students and administrators. They suggest how

each member of these groups within the organization should

behave, especially in interactions with others in different

roles. Teachers, for example, are expected to maintain

order in their classrooms and the school. School policies

are promulgated to govern such events as student absence,

tardiness, failure to complete assignments, when students

may use the rest rooms and lockers, etc. Policies describe

what teacher and student behaviors should be in each of

these instances. Less specified, but usually included in

the formal mission statement of schools, is another role

for teachers, one involving supporting the intellectual,

emotional, social and physical development of each student.

Both aspects of the teacher's role are expected to be played out in the everyday life of the school.

Although their actions toward each other are specified by their role, teachers and students encounter each other every day as individuals. Their first interaction may

include a role-specified definition of the other

{teacher/student) but the image each has of the other is 53 quickly enhanced by the particular characteristics of the other. (Is this teacher "nice", "mean", "strict", "laid back"? Is this student "bright", "b Io w ", "lazy",

"motivated"?) In addition, their encounter is influenced by their definition of their relationship. Will it be strictly role-based or will in include elements of the personal?

That is, will each individual reveal aspects of his/her subjectivity to the other? Will each make hiB/her personal identity available to the other in addition to the social identity prescribed through their respective roles?

In each encounter with a person who iB a student, therefore, the person who is a teacher has choices of which aspect of his/her teacher role to emphasize and has a choice of how much, if any, of his/her personal identity to reveal. For example, if a teacher encounters a student in the hallway during class time, the teacher can ask any number of questions of the student: Do you have a pass?

What is the trouble? Is there something the matter? I'm Ms.

Smith, what is your name? Why are you here now? Don't you know this is class time?

With her choice of questions the teacher reveals the image she has of the student (lost, in trouble, acting out, trying to get away with something) as well as the image of their relationship (whether or not it will be role governed, and if so, which role: teacher as enforcer of order or teacher as supporter of student well-being?). 54

The assumption that the teacher has the "right" to question the student upon seeing him in the hallway

(although the student does not have a similar right to question the teacher) reveals the deeper structure of the power relations in the organizational context. The conversation between these two individuals manifests the underlying power relations between teachers and Btudents in schools. That is, both the surface level text of the teacher's question and the underlying discourse rule of the context (which gave the teacher the right to aBk the question) reveal who has power in the organization.

The student does have some choices regarding the nature of the encounter, however. He can choose to play his role as rule abiding student (produce the hall pass for the teacher's inspection); or the role of student as Bupplicant

("I don’t have a pass but I have something in my eye and I need to get to the bathroom to wash it out."); or he can challenge the dominant power relations expressed in roles and policies ("It’s a free country and I can walk in the halls if I want to!"). Thus, each conversational encounter offers a fresh opportunity for individuals to reinforce or alter the dominant context of the school organization.

A closer look at how discourse affects organizational continuity and its role in facilitating change is provided in the next section. 55

Discourse/Conversation

As noted earlier, Wilden (1987) distinguishes between

the general set of signs available to a group of people

(language) and the specific words, subject matter and rules

for talking used by individual speakers (discourse).

Drawing on the work of Wittgenstein, the study of different macro and micro influences on the everyday talk of people

in groups has developed in sociolinguistics (Trughill,

1983), and the related disciplines of cognitive sociology

(Cicourel, 1973), ethnomethodology (Garfinkel, 1967), ethnography of speaking (Hymes, 1962, Duranti, 1988), and various combinations of these perspectives roughly referred to as the analysis of discourse (van Dijk, 1985).

Discourse is important to this study because through their conversations members reveal both their individual sense-making and how it is that they have collectively created their organizational reality. In addition, by changing their discourse, members can change their organizational reality. By following the discourse of the educators in Foster, I can follow their individual and collective sense-making about the state testing policy. If their conversations change to include, for example, developing an alternative response to the form of accountability inherent in the state policy, educators would be simultaneously changing their organizational 56 reality; that is, they would change how they act within the organization which thereby changes their social and symbolic construction of the organizational reality.

As people in organizations speak to one another about a given subject matter, there are rules operating to shape their conversation. The discourse rules emanate from the overall culture as well as from the specific organizational subculture, from the setting they are in at the moment, and from the underlying relationships of the conversants. In addition, the people talking are attempting to make sense, both individually and jointly, of the topic of their conversation. These mutually shaping elements of culture, setting, relationships, and sense-making combine to create a given context.

The context is dynamic; it is constantly created and recreated by the communicative acts of its members. But the member's creation of the ongoing context is affected by the rules which have emerged through the organization's history. As discussed previously, the actions and interactions of people in a group become habituated and institutionalized over time and are passed on to new members as if they were "objective facticities", rather than subjectively created by previous members. In short, members often do not realize or have forgotten that they created and can therefore change the rules which guide their actions and their talk. 57

Thia section briefly discusses some of the interacting rules shaping the discourse of people in a given context; it describes the rules generated from different settings, from personal and role relationships, and from their attempts at sense making.

The settings in which conversations occur influence the choice of words, tone of voice, and allowable subject matter Ivan Dijk, 1985). The same person, for example, will engage in different types of conversations at a concert, in a courtroom, or in a kitchen. Organizations have different settings in which their members interact. Part of learning the organization for new members is learning the rules for conversations in different settings. How doeB one talk during meetings? at the water cooler? or on the annual picnic? What topics are taboo in the conference room but allowed in the hallway?

Applying the notion of rule governed discourse to the interactions in schools, for example, an essential task for students is learning the teacher’s rules regarding how to participate in classroom conversations. That is, to be successful, students must recognize the teacher’s rules regarding how to talk, about what, to whom, when, and for what purposes (Green & Harker, 1982; Green & Weade, 1985;

Philips, 1985 ) .

The rules for conversations also apply to and generate from organizational roles and relationships. For example, 58 how does one address the company president? Who in the organization do you have to talk to first, second, third if you have an idea to present? As discussed previously, the enactment of formal roles are affected by personal relationships. When the president of an organization and her secretary talk with one another, their conversation is shaped by both their organizational roles and the personal relationship they have established through their past work together.

Conversations are also influenced by members' attempts to make sense of the work they are doing. Individuals

"accomplish" everyday organizational life by creating order and making retrospective Bense of what is going on around them (Morgan, Frost and Pondy, 1983). Weick (1977) used the term "enactment" to describe the proceBS in which people use symbols to make sense of and give meaning to their collective lives. The use of symbols, especially the symbols of language, thereby enacts the social reality of the organization. This ongoing process of creating meaning through conversation provides both stability for the organization and the possibility for change. With members constantly "organizing" through their sense making, the discourse serves to create, sustain, and, potentially, change the nature of the organization (Mumby, 1988).

As a specific case of sense-making, metaphors are an important part of the language repertoire of human beings. 59

Lakoff and Turner (1989) maintained that "far from being merely a matter of words, metaphors are a matter of thought" (p. xi ). Claiming metaphors are indispensable to both our imagination and our reason, they asserted that studying how people use metaphors provides insight into their culture aB well as their minds:

To study metaphor is to be confronted with hidden aspects of one's own mind and one's own culture...To do so is to discover that one has a worldview, that one's imagination is constrained, and that metaphor plays an enormous role in shaping one's everyday understanding of everyday events...It calls upon our deepest modes of everyday understanding and forces us to use them in new ways. (p. 214)

The notion that our metaphors reveal our worldview was echoed by Mumby (1988) who noted that metaphors actually

"shape the experience of social actors" (p. 18). He asserted certain organizational metaphors encourage certain organizational arrangements:

To oversimplify the relationship, military, metaphors can characterize highly structured, formal organizations, while family and organism metaphors might exemplify more flexible systems* (p. 18)

Pondy (1983) argued that using metaphors in conversations helps members bring meaning to their organizational experiences and helps them to resolve the inevitable paradoxes and contradictions of organizational life. By addressing the paradoxes, participants continually 60 engage in "organizing", that is, in creating the organizational reality.

The use of metaphors provides another important feature for organizations: metaphors simultaneously facilitate change and reinforce the traditional meaning given the organization. Pondy explained:

Metaphor facilitates change by making the strange familiar, but in that very process it deepens the meaning or values of the organization by giving them expression in novel situations....Because of its inherent ambivalence of meaning, metaphor can fulfill the dual function of enabling change and preserving continuity. (1983, p. 164)

Becoming aware of what their metaphors imply for what their organization is, as well as for what it might become, may be a first step for members to bring about change in the organization. If organizational members come to understand the metaphorical images used in their discourse they may also begin to see how they could construct a new image for their organization. Thus coming to understand their current organizational discourse, and then consciously choosing to change the discourse, are ways in which members can change their organizational reality. As

Evered (1983) claims: "Organizational change necessitates a language change" (p. 141).

In summary, the discourse of organizational members reveals how they are making sense of the work of the organization. The discourse shapes and is shaped by the 61 organizational context, particular work settings, and roles and relationships established by the members. The metaphorical images held by the members both reveal how they currently think about the organization and offer the possibility of conceiving of new ways of organizing.

As applied to this study, discourse is the key to understanding the sense making of the Foster educators in terms of the state testing policy; it is the means through which the researcher can uncover how the policy affects their ongoing interpretations of their roles, their relationships, and their work. Discourse is also the means to create a new organizational response to the state policy. If they choose to, Foster educators could articulate their local vision of education in response to the one implied in the state policy. Such conversations would clarify their image of what Foster schools are and how educators within them go about educating students. The next step would be to develop a form of accountability to appropriately assess their image of their schools. This study recounts the educators' current interpretations of the state testing policy and their frustration with not being able to articulate and establish an alternative way of being held accountable.

The role of external policies in affecting the actions of organizational members is presented next. 62

Policy

Within and outside of organizations* policies describe

the actions people are expected to take. Kerr (1976)

described policy language as an "action" language. That is,

in contrast to a "behavior" language, which includes

automatic and built-in reactions, action language focuses

on human intentions and purposes. Policies are

intentionally and purposefully written and promulgated to

affect the lives of others. This discussion begins with

past attempts to define the term 'policy* and considers the

various types of policies and their varying purposes. It

then describes different approaches to policy analysis and

specifically focuses on the issue of problem framing within

policy analysis. Attention then shiftB to the literature on

the implementation of public policies, particularly those

policies heralded as instruments of "school reform". Most

pertinent to this study, the discussion concludes with

consideration of educational testing policies, especially

state initiated testing policies.

Policy Definitions and Types

A number of scholars have described the difficulty of defining the term 'policy* (Dunn, 1981; Mitchell, 1984;

Prunty, 1984). Definitions range from very broad statements, such as "the conduct of public affairs or the administration of government", (Dunn, 1981, p. 7) or "what 63 governments do and say" (Dubnick &. BardiB, 1983, p. vii) to very detailed descriptions of conditions which must be met before one can say a policy exists (Kerr, 1976). The attempt to define policies reflects their ambiguous and multifaceted nature. Some scholars have tackled the task of definition by describing the different types of policies that exist and the different situations in which they would be called forth {Lowi, 1972; Prunty, 1984; Mitchell, 1984).

Mitchell claimed that the definition of policy emerges from "its role in resolving the two fundamental human conditions of scarcity and conflict" (1984, p. 138). Given that human social life is not characterized by unlimited resources, spontaneous cooperation, and shared values, policy decisions are needed to control conflict over scarce resources in society. Depending on one *s social science world view, policy can be viewed as exercising direct control or more indirect influence over the actions of groups and individuals. Such policy actions as regulations, incentives, and symbolic exhortations toward collective goals are all different devices for publicly affecting how resources are distributed in society.

Lowi (1972) presented four types of public policies, all of which more or less coerce individual and collective conduct. Although Lowi's taxonomy was also interested in demonstrating how the type of policy implies a particular form of politics to go along with it, of interest here is 64 in his distinctions among distributive, regulative, redistributive and constituent policies. While constituent policies have to do with setting up constituencies (e.g., reapportionment of voting districts), distributive and redistributive policies allocate or reallocate resources among different groups or individuals, and regulatory policies impose restrictions or limitations on the actions of individuals and groups (Prunty, 1984).

Thus the type of policy affects its definition. It may serve to control the actions of those affected

(regulatory); or to provide resources, such as funds, subsidies, programs, rightB, to particular individuals or groups (distributive); or to shift resources from one group to another, through, for example, the progressive income tax (redistributive); or to define which citizens fall into which voting units or are eligible for services from which public agencies (constituent).

For the purposes of this discussion, the focus is primarily on regulatory policies, which, whether promulgated by state or federal governments or within specific organizations, tell individuals and groups how to act in certain circumstances. In keeping with the definition offered by Kerr (1976) these policies reflect specific intentions and purposes of policymakers, and are thus a form of human "action", not just "behavior". In her philosophical treatment of the definition of policy, Kerr 65 specified four conditions for considering an action a policy. The last condition specified how the policy may be modified. Of interest here is the first three conditions because they directly relate to the state testing policy which is the focus of this inquiry. Examples of how this definition would fit into an educational context are inserted in brackets for illustrative purposes:

Condition 1. Some authorizing agent [e.g., state legislature] obligates itself to direct some implementing agent [e.g., local school board or district administrator] to act in accord with a specified conditional imperative.

Condition 2. The conditional imperative [e.g., administer state tests] must be in the form, Do some specified X-thing [e.g., testing] whenever, without exception, specified conditions [e.g., you work in a public school with grades 4, 6, 8, 9, or 12] occur.

Condition 3. The authorizing agent [state legislature] undertakes the obligation {in condition 1) for the purpose of effecting some state of affairs [e.g., improved schooling] and to do so without violating any restrictive rules [e.g., specific provisions for the testing] by which the authorizing agent [state legislature] would claim to abide {Kerr, 1976, p. 39).

Kerr thus provided a technical way of Baying that regulatory policies describe what people should do under given conditions in order to bring about some desired state. For the purpose of this study I will use this more informal definition of the term.

A fundamental aspect of any policy, however, is its role in the allocation of values. In seeking to bring about "a more desired state" all policies, whether distributive, redistributive, regulatory or constitutive, reflect such issues as power, control, legitimacy, privilege, equity, justice, and above all, values (Prunty, 1984). That is, the outcome regarding who will gain and who will l oBe in distributive, redistributive and constituent policies, and the rules promulgated within regulatory policies reflect the values of the policy makers.

Within education for example, as argued by Bernstein

(1971), curriculum policies reflect what counts as knowledge, pedagogical policies reflect what counts as a valid transmission of knowledge, and evaluation policies reflect what counts as a valid demonstration of what the learner knows. These policies reflect the values of those in power to legitimate certain forms of knowledge, transmission, or demonstration over others. In essence, as summarized by Prunty (1984):

...Policy is the 'authoritarian allocation of values'...To ask what counts as knowledge and culture in the schools is s I b o to query whose values have been validated. The authoritative allocation of values draws attention to the centrality of power and control in the concept of policy, and requires us to consider not only whose valueB are represented in policy, but also how these values have become institutionalised. ...On the bottom line, policy is the legitimation of values, (p. 42) (emphasis in original)

In asking such questions as whose values are reflected in a given policy, and how these values came to be 67 legitimated, Prunty described a critical role for the activity of policy analysis. The next section considers some of the ways in which policy analysis has been conceived with special attention to its role in framing the nature of the "problem" which new policies are designed to address.

Policy Analysis and Problem Setting

Definitions of policy analysis often use Lasswell's

(1977) distinction of policy sciences concerned with constructing knowledge o£ the policy making process versus knowledge used iii the policy making process (Dunn, 1981;

Mitchell, 1984). The aim is to distinguish between the study of the decision processes of the public order (which is more akin to traditional social science) from the scientific knowledge used within the formation of policy

(in which the knowledge becomes a political resource, that is, a part of the political debate) (Mitchell, 1984).

Dunn (1981) emphasized the practical nature of policy analysis, claiming that its fundamental purpose is "to provide policy makers with information that could be used to exercise reasoned judgement in finding solutions for practical problems" (p. 7). His definition of policy analysis focuses on this applied aspect of the inquiry: 68

Policy analysis is an applied social science which uses multiple methods of inquiry and argument to produce and transform policy-relevant information that may be utilized in political settings to resolve policy problems. (Dunn, 1981, p. 35)

Healy (1986), however, objected to thiB "received" view of policy analysis which attempts the "ideal of improving the lot of men [sic] in Bociety by employing the findings of social research to produce a better social order" (p.

383), Healy argued that neither social engineering nor instrumental rationality (which are assumed by this traditional view of policy analysis) provides the promised attainment of a better social order. That is, policy goals cannot be attained by manipulating causal variables in the social environment (social engineering); nor can the most efficient means of accomplishing these goals be scientifically determined by an ends-means analysis

(instrumental rationality).

In his argument against instrumental rationality, Healy agreed with Rittel and Webber (1973) that most contemporary social problems are by their very nature "ill-Btructured" and "wicked"; that iB, they have neither a unique best solution nor a unique best formulation. As a result, there are competing interpretations of both how the problem

Bhould be defined, or framed, as well as how it should be solved. Furthermore, pretending that "practical" science can decide the social goals or the social means of a policy 69 puts the value choices inherent to any policy in the hands of the technician (the policy scientist) rather than in the hands of the policy makers (Rein ft White, 1977).

Policy analysis, however, can serve some function in society, even if it cannot determine the one best way to solve a given social problem. WeiBs (1982) argued that policy research can affect "the shape and content of polity discourse, rather than concrete choices" (p. 623).

Referring to this "enlightenment" function, Weiss elaborated:

Research modifies the definition of the problems that policy makers address, how they think about them, which options they discard and which they pursue, and how they conceptualize their purposes. (p. 623)

By affecting the discourse of the policy, one can affect the policy itself. For example, by attending to the assumptions underlying the labels used for such characterizations as "at-risk children", "urban slums",

"drunken driving", one can affect the solution that is offered for the problem which has been implied in the label

(Gusfield, 1976, 1979; Rein ft Schon, 1977; Weiss, 1982). In effect, the policy solution is forecasted or implied in the way the problem is set or framed: "The questions we ask shape the answers we get" (Rein ft Schon, 1977, p. 236).

In keeping with the belief in the social construction of knowledge (Berger ft Luckmann, 1966; Blumer, 1969), Rein 70 and Schon openly acknowledged the importance of finding a

"frame" in which to make sense of the inchoate, uncertain and troublesome situation which creates the problem for policy makers. The frame we choose, however, will highlight certain aspects of the situation and ignore others. Similar to generative metaphors (Schon, 1979), these frames offer a way of looking at, or "setting" the problem, which in turn contains the prescription, or direction for the action we will take to solve, or address the problem (Rein Jfc Schon,

1977). Problem setting includes selecting the purposes we hope the policy to achieve and the means for achieving them

(Schon, 1979 ) .

Thus we are back full circle to the discussion of values; that is, who will gain from the purpose selected by policy makers and who will benefit or be harmed by the means chosen to accomplish the purpose (Prunty, 1984).

Healy (1986) maintained that policy research can be used not to socially engineer human beings (which denies their autonomous nature) but instead to help them seek self-understanding in order that:

...coming to see themselves and their situation in a larger, more informed context, the actors can themselves decide on the appropriate goals for [their own] future development, (p. 386)

In order for all human actors to decide for themselves how they might best develop, or accomplish some social ideal, the policy making discourse will have to be expanded 71

to include multiple voices, voices that will both help

shape the policy question and to formulate the solution.

This self-understanding or enlightenment through participation requires that not just the elite, but all the members of a group participate in determining who they are

and what they want to become. As noted by Fay (1976) in

contrasting the social engineering and the enlightenment

goals of policy research:

In the former an elite determines the rational courses of action for the group by knowing certain natural necessities, whereas in the latter all the members of the group actively engage in deciding what it is they are and want, and what arrangements must be altered or established in order to fulfill themselves", (p. 105)

The possibility of all the members of a group using policy research for self-understanding to jointly participate in the formulation of the problem and its policy solution requires free and uncoerced communication between policy makers and those affected by the policy. It also requires a focus on "situated rationality" (Healy,

1986). That is, given the multiple meanings constructed by different actors, and the need to make sense of their own situations in historical as well as current terms, the policy formulation needs to reflect the ways of knowing for that particular situation. Hence, understanding and appreciating the different contexts in which a given policy will be enacted by the different actors, makes it less 72 likely that one best policy solution (or even problem setting) will make sense for all contexts. This need to tailor the definition and solution of the problem to the subjective realities of the different actors furthers the necessity for the active participation of affected parties in the policy analysis process. An appreciation for the power of local contexts to shape how an external policy is enacted within an organization is the central finding of the literature on policy implementation.

Policy Implementation

The literature on implementation of public policies, especially policies sponsoring or mandating educational innovations, is replete with examples of how policy formulated at one level of government is not necessarily implemented as intended by the actors at local levels.

Although federal policy makers may have been indifferent to local context, the "street level bureaucrats" who are charged with its implementation, often without necessary conditions or resources, are aware of how a given policy needs to be adapted to local conditions (Weatherly &

Lipsky, 1977). As individual practitioners interpret, make sense of, and act on the policy, a "mutual adaptation" occurs in which the policy goals and regulations are fit into the local context (Bardach, 1977; Berman & McLaughlin,

1977, 1978; Elmore, 1977; Elmore & McLaughlin, 1983; 73

McLaughlin, 1976; Pressman & Wildavsky, 1984; Van Meter &

Van Horni 1984).

Furthermore, Richard Elmore and Milbrey McLaughlin's

analysis of the implementation literature suggested that

policy success depends upon both local capacity and local

will to carry out the policy (Elmore, 1983; Elmore &

McLaughlin, 1982, 1983; McLaughlin, 1987). In essence,

policymakers cannot "mandate what matters" (McLaughlin,

1987, p. 172). The essential aspects of getting a new

program started and sustained in a school depend on host of

local conditions (e.g., local stability, competing

authorities, opposing priorities) as well as the belief

systems of the individual people charged with changing

themselveB to implement the innovation (McLaughlin, 1987).

Furthermore, an emphasis on compliance with regulations

regarding the policy (such as the increasing specific

regulations promulgated for the compensatory education

programs of ESEA Title 1/ECIA Chapter I during the 1970's

and 80*s) does not insure that the programs are effective

in accomplishing their purposes. As McLaughlin (1987)

summarizes: "An army of auditors would be unable to force

compliance with the spirit of the law - which is what matters in the long run" (p. 173).

Unless the federal or state policies are aligned with

or reinforce current local efforts (Firestone & Bader,

1989; Odden & Marsh, 1988), they generally get implemented 74 only after a long and slow process of adaptation

(McLaughlin, 1987).

Another lesson learned from reviewing the policy implementation literature, is that when policies call for a change in attitudes, beliefs, or routine practices

(characteristics associated with school reform policies), a combination of pressure and support is needed from the policy. Pressure is needed to focus district attention on the reform goal; without such external pressure other priorities and demands will take center stage. Support is needed to enable the implementation to occur. Teacher training, consultants, and materials, are some of the enabling conditions needed for implementation of innovations (McLaughlin, 1987).

A final aspect of the general policy implementation literature is the notion that change ultimately occurs at the smallest unit of the organization: "Organizations don’t innovate or implement change, people do" (McLaughlin, 1987, p. 174). Professional and personal motivations combine with organizational incentives to affect how the individual actor responds to the policy initiative. A focus on the response of those most affected by and closest to the level in the organization where the implementation occurs is thus essential to understanding if and how policies made external to the context of individual practitioners become incorporated in their daily practice. 75

School Reform Policies: Top-down and/or Bottom-up

As noted at the outset of this discussion on policy,

policies are written with the express purpose of affecting

the actions of others. In the last decade state

legislatures have promulgated hundreds of educational

policies to bring about "school reform" (Pipho, 1986).

Whether or not these mandates have actually helped the

education of students is the subject of much controversy in

the educational literature. As Mitchell (1984) argued,

although values shape the policies which get written, the cultural and material aspects of the school determine whether these policies will influence practitioner performance:

Policy makers cannot teach students, and they cannot manage school programs unless they change jobs and join the school Btaff. Hence they must find policy mechanisms that may be used to indirectly restructure the school system through influencing the actions of educators and students by changing the cultural and material environment in which they operate, (p. 154) (emphasis added)

Finding the right policy mechanisms in which to change

the cultural and material environment of the schools is the most difficult part of state attempts at school reform. A

review of the school reform legislation reveals that it

includes such provisions as increases in teacher salaries, expanded kindergarten and early childhood programs, teacher aides for lower grade classrooms, entry year programs for 76 new teachersi career ladders and/or merit pay for teachers, staff development for teachers and administrators, and incentive programs for teachers, schools and districts

(Pipho, 1986). These aspects of reform could be considered attempts to build the local capacity of practitioners.

The reform legislation has also included, however, elements focused on controlling or regulating aspects of the school environment and the people within it, for example: state standardization of local curriculum, increased state standards for graduation, statewide competency tests for students, competency testing and required evaluations of teachers and administrators, state policies for the placement, retention/promotion and remediation of pupils, changes in certification requirements, and changes in length of school day and year

(Pipho, 1986).

The question for educational policy analysts has been whether the regulatory aspects of state initiated reform might do more harm than good; that is, will attempts at controlling the teaching-learning process from afar (i.e., from the statehouse) backfire and actually make matters worse for the educational experience of students? The essential argument of these researchers can be summarized as follows: Given that the moBt important aspect of education is the interactions between teachers and students, the way to improve schooling is to improve teacher capacity to make wise curricular and instructional

decisions in the moment-to-moment, day-to-day practice of

the science and art of teaching. Such "bottom-up" policy

support for a culture of professionalism among educators

with its inherent property of their making the curricular

and instructional decisions on behalf of students is

contrasted to top-down efforts to exercise control over

curriculum and instruction and have teachers and

administrators simply implement the decisions made by those at some distance from the schooling process. This top-down,

regulatory control is seen as undermining the quality of

interactions between teachers and students and thereby worsening, rather than improving, the educational process

(Boyd, 1987; Conley, 1988; Darling-Hammond, 1988; Elmore,

1983, 1987; Elmore & McLaughlin, 1982, 1983; Hawley, 1988;

Kirst, 1988; McLaughlin, 1987; McNeil, 1988 a, b, c;

Passow, 1988; Shulman, 1983; Soltis, 1988; Timar & Kirp,

1988, 1989; and Wise, 1979).

In effect, the attempts to legislate learning (Wise,

1979} from the statehouse undermine the essential and legitimate authority relations between teachers and students in the classroom (Elmore, 1987). When teachers are seen by students as mere functionaries, carrying out decisions made by someone elBe, the eBBential quality of the teaching/learning interaction (in which the student gives authority to the teacher to teach on the basis of the teacher’s knowledge and genuine commitment to student learning) is eroded (Elmore, 1987). As a consequence, educational content is fragmented and reduced to narrow

"lists" of facts which teachers lecture from for ease of presentation, students memorize for ease of learning, and schools and states use in multiple choice tests for ease of assessment and accountability. Teachers do not bring their personal passion for the subject matter to their students; controversial topics are kept out of classroom discussions; the emotional environment of the classroom is flat; teachers and Btudents make a "bargain" not to disrupt the work lives of the other; and neither are intellectually nor emotionally engaged in the educational process (Elmore,

1987; Goodlad, 1984; McNeil, 1988b; Powell, Farrar, &

Cohen, 1985).

This unsettling scenario is attributed by Boyd (1987) to reform minded policymakers’ inadequate understanding of the politics and organizational character of schools.

Because they do not take into account the culture and organization of schools, Boyd argues, top-down reform mandates can backfire and bring about "the laBt hurrah" of public education, rather than its renaissance. As current support for public schools weakens through demographic changes (less people have children in schools) and disaffection (middle class and affluent parents move their children to private schools), there is not much time to 79

reinforce the need for and past success of the public

school as a major institution in a democracy.

What is needed now is policies which support a balance

of control and autonomy in schooling. Recognizing that

equity and excellence concerns will not be realized by a

complete reliance on laissez faire local control (the need

for Brown v. the Board of Education and compensatory

education has taught us this lesson); and that excessive

centralization and regulation will not bring about the type

of student-teacher interactions that further intellectual

and emotional growth, we must find the right balance

between top-down leadership at the state and federal levels which set key values and goals, and bottom-up support for teacher capacity building (Boyd, 1987). Such support would encourage, and not undermine, the professional expertise of teachers to wisely make the day-to-day discretionary choices most important to the instruction of students and the improvement of schooling. Since legislators cannot mandate what matters, they need to find the policy tools which support those who do matter in their exercise of professional judgement (Darling-Hammond, 1988; Elmore,

1983; McLaughlin, 1987; Soltis, 1988).

The final aspect of educational policy to be addressed in this review considers one prominent part of school reform legislation, state mandates to test student achievement in schools. 80

State Testing Policies

The use of state mandated tests as an educational reform tool has proliferated in the last decade (Afrasian,

1987; Pipho, 1986) and has been a source of some controversy within education (Airasian, 1987, 1988;

Ellwein, Glass, & Smith, 1988; Ginsberg & Berry, 1990;

Haney & Madaus, 1989; Livingston, Castle, & Nations, 1989;

Neill & Medina, 1989; Nickerson, 1989; Pearson & Valencia,

1987; Popham, 1987; Purkey & Smith, 1983; Schujaa, 1989;

Shepard, 1989; Wiggins, 1989; and Wise & Darling Hammond,

1983). The works of Peter Airasian (1987) and George Madaus

(1985) provide a useful synthesis of the overall dilemmas associated with the state testing policies emanating from the school reform movement of the 1980s.

Numerous reasons have been given to the increased reliance on paper and pencil, multiple-choice indicators of student knowledge. Airasian (1987) claimed that tests are trusted and desired by the public because they are symbolically linked to preservation of "standards" and to traditional (that is, not progressive) values in education and they are perceived as fair (because all students take the same exam), "scientific" (because they produce a number), and objective (because they are not influenced by the bias of teachers, principals or parents). In addition, they provide some sense of state control over local practice. 81

In the past, local districts were the primary users of standardized tests. They used tests to select students for special programs, to group students, and to identify

Btudent needs. In addition, states assessed a sample of students with these tests to monitor the educational system as a whole. The new wave of state mandated testing associated with the school reform movement of the 1980's, however, shifted the purpose of testing from its local use of guiding instruction and its previous state use of monitoring education to the new state use of motivating school reform (Airasian, 1987; Madaus, 1985). Airasian provided three summary characteristics of such state mandated testing:

1. They are mandated by state legislatures or state boards of education for all school districts and for all or virtually all members of groups at selected grade or occupation levels. The purpose of the test is to certify the performance of each district or individual.

2. State mandated certification testing programs eliminate most local district discretion by using a single statewide testing instrument that is administered, scored, and interpreted similarly from district to district according to state guidelines.

3. There are clear sanctions and rewards associated with test performance. The test have a built in requirement and criteria for making a decision on the basis of performance on them. (1987, p. 404) (emphasis added)

With this new emphasis on state control of what the test will measure and how students and districts must 82 perform to receive the rewards and avoid the sanctions

imposed, state testing became associated with the political process of distributing resources. With tying state monies to test scores or determining, for example, which students will receive a diploma (heretofore a prerogative of the local district), the state testing program became intertwined with the social, economic, and political issues of control and equity.

Airasian provided a framework of 14 propositions for understanding the current context of state testing programs. The following eight propositions are most salient to the present study:

a. Testing takes place in a politicized environment in which the agendas of different interest groups compete for attention.

b. The crucial issueB of testing are not technical. Issues of testing today are social, economic, and value laden, involving the distribution and redistribution of resources and prerogatives.

c. The quantity of educational testing is inversely related to the public’s general satisfaction with the educational system.

d. The political benefits of testing often outweigh the educational benefits.

e. Two important functions of testing in the policy domain are management and control.

f . Most tests supply redundant information. What is important politically is that tests appear to supply it in an objective, standard manner.

g. The greater the consequences associated with the test the greater the response to the test and the greater the influence of the test on practice. 83

h. Given the important consequences that ensue from policy-oriented testing, not to teach to the test may be a greater disservice to pupils than to teach to it. (Airasian, 1987, pp. 408-409)

Given this framework for understanding the background and consequences of state mandated testing, it is easy to appreciate how testing has become a policy, rather than an

instructional, tool in the educational community. "High stakes testing" refers to the rewards and punishments associated with scores obtained on the tests (Airasian,

1988; Popham, 1987). According to Madaus (1985):

Attaching important rewards and sanctions to the results of tests transforms testing into a coercive device that can influence curriculum and instruction. Under these circumstances, testing becomes a classic example of winning through intimidation, (p. 614)

Madaus outlined some of the high stakes for school districts when state policymakers use tests to "make things happen". Policy makers in various states have linked the following provisions to performance on state tests:

-performance contracts for districts to access state funds;

-compensatory education funding;

-certification of each student as meeting minimal or maximum competencies;

-merit pay for teacherB based on student scores (an "educational Frankenstein", according to Madaus). (1985; p. 614) 84

Within this approach of "using test scores as goads", state policy makers see other advantages to testing policies: they can motivate lazy or recalcitrant students to work harder to get a diploma; they can be used as a weapon (notice the metaphor) to explain to parents why their children are not being promoted or graduating; and symbolically they serve to give policy makers the appearance of doing something about education (Madaus,

1985) .

Test scores have the symbolic quality of an "objective, scientific, and technological totem"; as a pseudo scientific myth they serve as "the amazing grace to save the world from illiterate graduates" (Madaus, 1987, p.

615). In addition to bullying the educational community into line, they replace teacher judgements as the decision criteria for student promotion or graduation.

According to Madaus, the consequences for the educational system of using tests as a policy tool are numerous:

1) shift from local to state control of the content and methods of teaching (the level in the Bystem which controls test content controls curriculum and instruction);

2) a narrowing of the curriculum to what is measured on the test;

3) a shift in teaching practices to essentially drill for the test; and

4) in addition to the intellectual and emotional toll of the testing itself (especially for the poor and minority students most adversely affected) all students 85

coached for the test face the possibility of passing the exam without receiving an education. (Madaus, 1985)

In summary, Madaus concludes that "in mandating tests, policy makers have created the illusion that test performance is Bynonymous with the quality of education"

(p. 616). Educators take exception to this reductionist definition and insist, if we must have tests, let them assess what we do teach (Wiggins, 1989). There is a call among educators for development of new, authentic measures, such as the "exhibitions of mastery" proposed by Sizer

(1984) or the performance assessments being considered in a number of states (Baron, 1989; California Assessment

Program Staff, 1989).

A critical aspect of the current state testing programs is the removal of teachers from the instructional decision making process (Pearson & Valencia, 1987). When the state tests do not match the content of the local curriculum nor the instructional methods used by teachers, and yet teachers are held accountable for test results the professional culture of schools is being denied. As noted earlier, the attempt to "reform" schools while ignoring the cultural and organizational contexts in which such reforms must be carried out may evoke compliance but will not guarantee an improvement in the education of students.

Thus state testing policy represents a powerful example of the regulatory/controlling aspects of the school reform 86 movement. Rather than enhance the capacity of or "empower" educators to take on additional responsibilities and decisions, it strips educators of the most crucial decisions in education: what and how to teach. The struggle between these two conflicting views of how to change schools is being played out in the daily life of educators across the country. The next section focuses on how each of the elements of policy, organization, and language relate to shape the context and discourse of a particular school district.

Policy, Organization and Discourse

in a School District

As described above, the effect of some Btate policies can be to undermine rather than enhance the authority of teachers in the classroom (Elmore, 1987). Madaus (1985) reminded us that testing policies can result in students passing the test without learning. He quotes Cincinnati superintendent Emerson White’s description of the dilemma for teachers when, in 1888, they were faced with a test used for Btudent promotion:

In the very nature of things, the coming examination with such consequences must largely determine the character of prior teaching and studying. Few teachers can resist such influence... They Bhut their eyes to the needB of the pupils and put their strength into what will "count" in the examination. (White, 1888, p. 518) 87

The consequences, for students and educators alike, of not teaching to the test are grave. Yet the consequences of teaching to it, namely the potential loss of an education, are grave as well. Just how a state testing policy impacts a particular school district is the Bubject of thiB

inquiry. Whether these educators see a conflict between

the needs of the students and what will "count in the examination" is one of the questions for this analysis of the policy.

In this case study the "policy analysis" is focused on how local actors make sense of, or interpret, the state testing policy within their particular organizational context and its inherent culture. This line of inquiry is in keeping with the recent call for policy analysis which is sensitive to context and focuseB on the meaning-making practices of educators, particularly administrators

(Anderson, 1987; 1990; Foster, 1989; Goodlad, 1988-89;

Wimpelberg, Teddlie & Stringfield, 1989).

Sensitivity to context requires an understanding of the cultural norms of the organization being studied. Meryl

Reis Louis (1983) emphasizes that organizations are

"culture bearing milieux"; that is, they are distinctive social units with common understandings regarding how members should act. Organizations embody certain ideals which serve the group in multiple ways: they represent what the group aspires toward; they sanction behavior; and they 88 are used to evaluate performance of members. These social ideals also provide a sense of group identity:

A cultural view [of organizations] encompasses the system of social ideals and the B e t of symbolic devices (i.e., myths, rituals, signs, metaphors, special languages) that embody and are used to convey ideals. ...Culture embodies the identity of the social group. What we, as members, stand for and how we deal with one another and with outsiders is carried in and through culture, (pp. 43-44)

Uncovering what it is that a group stands for is thus crucial to determining its identity. In studying the cultural identity of an organization, however, it is common to find that within a single organization there are multiple, nested and overlapping cultures. One task for the researcher, then, is to determine just which subculture is being investigated (Louis, 1983). Each of the possible groups develops its own ethos, or distinctive character, which its members use to both frame their view of the world and their "image" of themselves in the world. In this study the subculture of interest is composed of a set of professional educators in the school district, namely central office administrators responsible for curriculum and instruction (as opposed to public information, facilities, and finance, for example), building principals, assistant principals and some of the teachers who were engaged in conversations about the state testing policy. 89

Erickson (1987) points out that within educational organizations, part of what is included in the professional educators' cultural view of the world is what counts as knowledge to be taught in school. He cites, for example, the difference between the United States and Japan in educators' views toward the teaching of reading. In the

U.S., reading is treated as something very difficult to learn which therefore absorbs a great deal of schooling time and attention. In Japan, however, reading is viewed as something relatively easy to learn which the child will most likely learn at home and in the community. The difficulty comes with learning to think well, so teaching time and energy is devoted to the development of judicious reasoning. Additional studies have described how different views of what kind of knowledge is important can be held by teachers in different schools (Anyon, 1981; Corbett,

Firestone & Rossman, 1987).

Part of the cultural identity of educators in a given school, therefore, is their definition of what knowledge is important to teach. A critical factor in bringing about change in an organization is whether the proposed initiative confronts an aspect of the culture which is central to its identity. Building upon the work of Sarason

(1971), Corbett, Firestone & Rossman (1987) distinguish between "sacred" and "profane" cultural norms in schools.

The distinction rests on whether the norms are open to 90 change: " ‘the sacred* characterizes immutable norms; and

’the profane* those susceptible to change" (p. 37).

Within schools the sacred norms are those tightly- associated with professional purposes, with educators* raison d ’etre. Any change which challenges the professional identity of the teachers or administrators is likely to meet the most resistance because it is seen as an assault on what is sacred in the organization, on what culturally defines them as an entity and provides their personal reasons for being educators:

Attacks on the sacred undermine professional identity and call the meaning of teaching into question. Proposed changes, then, challenge more than "the way we do things around here" (profane norms]; they also threaten "who we are around here" [sacred norms]. (Corbett et al ., 1987, p. 56)

For the educators in Foster, the state testing policy may threaten a "sacred" norm, that is, their definition of what the local curriculum and the local form of instruction should be, and what they define as "best for students".

Similar to the staff in one of Corbett, Firestone & Rossman

(1987) case studies, the Foster Btaff may be "unable to consider, much less approve of, decisions based on criteria other than learning - such as public relations, legalities, or political considerations" (p. 43). That is, even though scoring well on state tests is important for public relations, legal, and political purposes, these criteria 91 are insufficient for these teachers to teach to the test.

As we shall see in Chapter IV, the state testing policy challenges a sacred cultural norm in this district, the underlying definition of what it means to he an educator within this particular organization.

As noted earlier, one of the expressions of a cultural context is the members’ rules for conversing with one another. To elaborate on the statement from Louis {1983) referring to cultural identity as associated with "how we deal with one another and outsiders" (p. 44), cultural identity also proscribes "how we talk with one another and outsiders". The rules for discourse in an organization both shape and are shaped by its culture (Bateson, 1972;

Goffman, 1974; 1983; Hall, 1976; and Wilden, 1987). The discourse rules for the conversations surrounding this state policy are therefore of primary interest to the study. How these educators talk to each other within their cultural context and how they talk to "outsiders"

(including the researcher and members of the school board) is a central concern of this study. Prior to focusing upon the findings regarding policy, organization and discourse, however, we turn to a description of and rationale for the research methodology used in this inquiry. CHAPTER III

RESEARCH METHODOLOGY

Introduction

In this chapter both the philosophical basis for the research methodology and the particular inquiry methods used in the conduct of this case study are presented. A general description of the interpretive and critical research paradigms and their attendant methodologies is provided first, followed by a rationale for the study’s focus on the practitioners conversations. In addition, the central role of the person of the researcher in this type of inquiry is addressed as well. The research methods and the analysis process are then described. The chapter concludes with a discussion of ethics, reflexivity and the perceived limits of the study.

Interpretive Paradigm and Naturalistic Inquiry

This inquiry is rooted in the interpretive paradigm of the social sciences. Its focus is on developing an understanding of human events and actions, that is, on what meaning can be made of human actions and interactions

92 93 in their socially constructed world (Berger & Luckmann,

1966; Blumer, 1969; Burrell &. Morgan, 1979; Erickson, 1986;

Morgan, 1983). The interpretive paradigm is based on the philosophical traditions of phenomenology (e.g., Heidegger and Husserl) and hermeneutics (e.g., Gadamer, Heidegger and

Ricoeur). Essentially its methodology centers on the phenomenological concern for how reality is constructed and experienced through the conscious acts of humans and the hermeneutic concern for interpreting meaning in lived experience as well as in written text (van Manen, 1986).

To seek an understanding of human meaning in social life requires the researcher to examine directly the actual group life of the people in question. With the assumption that the social reality of any group is socially constructed through the symbolic acts of its members,

Blumer offered four methodological implications for the study of group life :

1. Given that people act on the basis of the meanings of the objects that comprise their world, the scholar who wants to understand the actions of people, must see their objects as they see them (i.e., from an emic, or insider's, perspective).

2. Given that people act in accordance with how they see and interpret the actions of others, the scholar has to see how members form their lines of actions on the basis of others' actions. This process is always ongoing, and cannot be understood by compreBBing it into some predetermined social science category. 94

3. Given that social action is constructed by actors* that is, when confronted with a situation in which they must act, they note, interpret, assess, and develop a line of action, the scholar must observe the process by which the action is being constructed.

4. Given that large scale organizations are not entities operating in their own right with their own dynamics, merely expressing the invisible forces of the overall system, but rather are arrangements of people who are interlinked in their respective actions who are constantly in the process of interpreting situations and acting based on their interpretations, scholars must focus on explaining the way in which participants define, interpret and meet the situations in an organization. (1969, pp. 50-60)

The methodology particularly suited for understanding

group life is naturalistic inquiry (Lincoln & Guba, 1985)

which includes aspects of ethnography (from anthropology)

and ethnomethodology (from phenomenological philosophy,

sociology and linguistics). The central feature of this

methodology is the researcher's focus on individuals and

groups in their natural context and the development of an

understanding of how these people make sense of and enact

their daily lives (See, for example, Bogdan & Biklen, 1982;

Fetterman, 1989; Geertz, 1977; Lincoln & Guba, 1985; Van

Maanen, 1988; Wax, 1971).

The definitions and descriptions of ethnography provide

insights into the work of the naturalistic inquirer.

Defined simply by Fetterman (1989) as the "art and science

of describing a group or culture" (p. 11), the focus of

ethnography is the culturally shared, common sense 95 perceptions of everyday experiences of members (van Manen,

1986). Clifford Geertz offered a perspective on both culture and the way it should be studied:

Believing with Max Weber, that man is an animal suspended in webs of significance he himself has spun, I take culture to be those webs, and the analysis of it to be therefore net an experimental science in search of law but an interpretive one in search of meaning. (1977, p. 5)

Geertz described ethnography, using Gilbert Ryle's term, as "thick description", that is, a description of the many complex, conceptual structures used by social groups to make sense of their everyday actions and interactions.

The task for the ethnographer is "first to grasp and then to render" (1977, p. 10) these meaning-making structures, or webs of significance. As Buch, the data for the researcher is the interpretations of the social actors, or more precisely, the researcher's interpretations of the members' interpretations. As Geertz describes it: "Data are really our constructions of other peoples’ constructions of what they and their compatriots are up to" (1977, p. 9).

A related focus for the ethnomethodologist is the way in which people in a group "accomplish" their everday life, abiding by taken-for-granted and largely unnoticed "rules" which underlie their discourse and interactions. In their examination of of group life, ethnomethodologists look for the unrecognized rules which shape members’ commonsense 96 understandings of everyday events (Bogdan & Biklen, 1982).

The case study is a common tool of naturalistic inquiry, ethnography and ethnomethodology. It iB essentially a "detailed examination of one setting, or one single subject, or one single depository of documents, or one particular event" (Bogdan & Biklen, 1982, p. 58). Yin

(1984) provided a more technical definition:

A case study is an empirical inquiry that:

- investigates a contemporary phenomenon within its real life context; when

- the boundaries between the phenomenon and the context are not clearly evident; and in which

- multiple sources of evidence are used. (p. 23)

The concern of those who question the usefulness of case studies in scientific inquiry is whether the results from case studies are "generalizable", that is, generalizable in terms of the scientific aim within the positivist paradigm of seeking lawlike generalizations

(Lincoln & Guba, 1985). Geertz's responded that the essential task of theory building within the interpretive paradigm "is not to codify abstract regularities but to make thick description possible, not to generalize across cases, but to generalize within them" (1977, p. 26). That is, the building of a theory to explain and understand human action relies on the detailed description of the actions of people within the given context and cultural 97 norms of their constructed reality. As Cusick (1973) maintained, "reasonable behavior for one normal human being in a situation is reasonable behavior for other normal human beings given that same situation" (p. 218). The naturalistic inquirer attempts to conduct a case study in such a way as to make it possible for the reader to vicariously understand the reasonable actions and sense making of the people within it (Donmoyer, 1988).

The extent to which the sense making of particular actors in a particular context seems reasonable to transfer to other contexts depends upon the similarity of the two contexts (Lincoln & Guba, 1985). With regard to this study, the findings related to how Foster educators make sense of the state testing policy will be relevant to other educators in so far as the actors and contexts of other school districts are similar to the actors and context of

Foster.

Critical Paradigm and Methodology

Essentially this inquiry is a case study of the actions and interactions of a Bet of professional educators within the context of a specific school district who are responding to a new testing policy initiated by state level policy makers. It is largely an account of the personal and collective sense making of some administrators and teachers within the cultural norms of their constructed

I 98

organizational reality. As such, it is primarily an

interpretive study, focused on the meaning making of these particular social actors. There is, however, an element of critical social science in the conduct of this inquiry. In the sense that there is some limited effort on the part of the researcher to act as a catalyst for bringing about self understanding and to ask actors why they interpret the policy as they do and what the possibilities for future actions might be, this study represents a modest effort within the critical paradigm.

The critical social science paradigm is concerned with asking how it is that individuals and groups come to interpret and construct social reality as they do. Rather than focus only on how members make sense of their social worlds {the focus of the interpretive paradigm), the critical scientist askB what psychological, historical and material conditions support this particular Bense-making

(Deetz & Kersten, 1983).

The translation of the critical paradigm into research practice has taken many forms. It is variously referred to, for example, as:

- critical social science (Carr & Kemmis, 1986; Fay, 1987) ;

- critical ethnography (Anderson, 1987, 1989, 1990; Gitlin, Siegel & Boru, 1988; Simon & Dippo, 1986); 99

- research as praxis {Lather, 1986; 1988) and emancipatory, participatory, educative research (Fay, 1977; Maguire, 1987);

- a critical approach to interpretive research (Deetz & Kersten, 1983);

- action science (Argyris, Putnam & McLain, 1985; „ Krim, 1988; Torbert, 1981);

- interpretive policy research (Healy, 1986); and

- new paradigm research and collaborative research (Heron, 1981; Reason & Rowan, 1981a; Reason, 1988; Torbert, 1981).

In essence, the research becomes an opportunity for members of the group being studied to further their self-understanding and ask if they might construct their future differently than their past. Such research has a

"catalytic" quality (Reason A Rowan, 1981b). In Lather's

(1986) words, catalytic validity refers to the degree to which the research process "reorients, focuses, and energizes participants toward knowing reality in order to transform it, a process Friere (1973) terms concsientization" (p. 272).

Although this case study has not dealt with the external historical, social, ideological and material forces that are commonly associated with critical theory, it does take the first step of encouraging self-reflection and deeper understanding through a focus on the conversations of the participants and how these conversation reflect their own sense-making. Deetz A 100

Kersten (1983) maintained that the three tasks of the

critical researcher are understanding, critique and

education:

Understanding requires descriptions of the social reality in the organization and forces that form, deform, sustain, and change that reality. Critique focuses on examining the legitimacy of consensus and reason-giving activities in an organization and the forces bearing on them. Education develops the capacity of organizational members to engage in self-formation through participation in organizational practices that are free and unrestrained, (p. 148) (emphasis added)

As described in the later portions of this chapter and

in the findings presented in the subsequent chapter, the

current study makes a modest attempt at each of these

tasks. It helps to build understanding by writing and

circulating descriptions of organizational members’ beliefs

and practices surrounding the implementation of the state

testing policy; it encourages critique by focusing on the

rules of discourse and interaction that seem to govern the

conversations and practices of the professional educators;

and it attempts to educate by offering possibilities for

new conversations for the development of an alternative

organizational response to the state testing policy. In

this effort to encourage participants to consider the

possibilities for enacting a different future, rather than

just reacting to what the state policy demands of them

within their current situation, this study is a very

* limited attempt at research as praxis (Lather, 1986). 101

The tightrope the critical researcher walks consists of providing participants occasions to view their own situations without the research also being exploitative. In effect the researcher must be prepared to assist participants critique their context without imposing the researcher's own critique, which would constitute just another form of exploitation (Gitlin, Siegel, & Boru, 1988;

Lather, 1986, 1988; Stacey, 1988).

Torbert (1981) asserted that researchers could actively assist practitioners if research would focus on practice:

Practitioners are generally attempting to act well in situations which they do not fundamentally comprehend, in pursuit of purposes which are not initially fully explicit and to which their commitment is initially ambivalent, and they are being interrupted all the while by other claims on their attention. ...[W]hat practitioners really require is a kind of knowledge that they can apply to their own behavior in the midst of ongoing events, in order to help them inquire more effectively with others about their common purposes, about how to produce outcomes congruent with such purposes, and about how to respond justly to interruptions (p. 143). (Emphasis in original)

The critical aspect of the present study involves assisting the educational practitioners in this setting understand how the state testing policy might be used to clarify their definition of their common purpose; that is, the testing policy might serve as an impetus for them to articulate their vision of education to their local community. The difficulty inherent in such a research 102 attempt is to provide assistance without imposing the researcher's perspective on the practitioners {Gitlin,

Siegel, & Boru, 1988; Lather, 1986; 1988).

Focus on Conversations

As noted above, the focus of the inquiry is on participants' conversations. Much of the work of critical research depends upon the Habermasian contention that social change can be accomplished through the establishment of free and open communication situations. Within these

"ideal speech situations” the forces which coerce and block emancipation can be recognized; such recognition of these forces is seen as a first step to changing them (Deetz &

Kersten, 1983).

Thus the interest in discourse for this study is two-fold. It first provides the means for understanding the meaning-making of the professional educators in Foster.

Within the interpretive paradigm, a focus on the talk of the participants provides a way of uncovering their social and symbolic construction of the reality of their organization and their response to the externally initiated policy which they must now make sense of and incorporate within their personal and collective belief systems.

Secondly, within the critical paradigm, a focus on the talk provides the possibility for change. With a new understanding of the situation, a new vision can be 103

imagined and articulated. As described in the next chapter, the educators in this study are dismayed at their inability to articulate the nature of the educational experience they are providing their students. The research process can provide the occasion to begin Buch an articulation. As

Deetz & Kersten maintained: "Research is important to human choice making" (1983, p. 171). Becoming aware of what some of their ongoing choices have been and what their future choices might be can enable these educators to choose to enact a different future. Such a future will come about, however, partially through changing the conversation. As

Evered claimed: "Organizational change necessitates a language change" (1983, p. 141).

The conversational focus therefore provides the data for the study and a method of analysis (van Dijk, 1985) as well as occasions to evoke action. To do so, however, requires the researcher to be a part of the conversations.

Heron (1981) described an active role for the researcher as essential:

To explain human behaviour you have, among other things to understand [symbolizing] activity, and fully to understand it involves participating in it through overt dialogue and communication with those who are engaging in it. (pp. 23-24)

Being a part of the dialogue not only insures that the researcher understands the meaning-making of the participants, it also insures that the participants have a 104

say in the researcher’s construction of the description of

their collective selves. Acknowledging with Simon & Dippo

(1986) that data are "produced" and not "found", it is

essential for the researcher to share her production of the

community’s construction of their reality, especially if

the research is to serve an educative purpose. Gitlin et

al. (1988), maintained that educative research "encourages

community members to argue out the interpretation of

reality and come to a mutual understanding" (p. 28). The

purpose of such research is to engage in a common effort to

understand and transform reality and "the image of the

solitary researcher trying to study a subject is replaced by one of a community bonded by discourse and action"

(Gitlin, Siegel, & Boru, 1988, p. 28). Thus the researcher not only studies but is actively engaged in the discourse and action of the members of the organization.

Another feature of this focus on conversation is its evocation of stories, or narratives, from both the participants and the researcher. Mishler (1986) reported that when the typical power relationships of researcher-subject are removed from the interview process and the "respondents" (not "subjects") are freed from the controlling aspects of the interviewer’s questions and allowed to organize their thoughts in their own ways, they tell "stories": 105

One of the significant ways through which individuals make sense of and give meaning to their experience is to organise them in a narrative form. ...In sum, interviewing practices that empower respondents also produce narrative accounts. There is, however, an additional implication of empowerment. Through these narratives people may be moved beyond the text to the possibilities of action. That is, to be empowered is not only to speak in one's own voice and to tell one's own Btory, but to apply the understanding arrived at to acting in accord with one's own interests. (Mishler, 1986, p. 119)

Thus the occasion to find and use one's voice to make sense of what is going on around us, also offers the possibility of evoking new action, in effect, to engage in speaking the language of possibilities.

Storytelling has also become important for the researcher. For this study, the most compelling way for me to convey the findings has been to organize and express them in three different, yet related, "stories". The first story tells what the educators in this study Baid and felt about the state testing policy; in other words, it is made up of their expressions about the policy. The second story tells about my experience of doing the research in their midst; it is my expression of the research process. The third story tells of their dilemma with how the state testing policy holds them accountable and includes my efforts to use the research process as a way to activate them to develop an alternative response to the policy. It concerns whether they might develop a new collective expression regarding the policy. 106

The notion of expression is critical to the idea of storytelling. Until now the justification of this inquiry has been on its explanatory power, that is, its ability to explain how a given group of people make sense of and enact their collective lives within a specific organizational context. The role of expression in inquiry is also important, however, in any accurate rendering of human social life.

Reason and Hawkins (1988) carefully articulated the distinction between two related paths of inquiry, one through explanation and the other through expression. Both paths start with experience and seek understanding. In essence, the researcher traveling the path of explanation starts with experience and uses records, data, and accounts to uncover patterns and themes, and with the help of case studies, eventually develops a typology and then a general theory to understand and explain the experience. The other route, one of expression, also begins with experience and travels through records, data, and accounts but seeks out metaphors, stories, sagaB, myths and fairy tales, and eventually develops archetypes rather than theories as the means to understand and express the experience.

On the path of explanation, the researcher can be detached, standing back and analytically relating what is found to the categories which make up his/her theory. On the path of expression, however, the researcher is deeply 107 engaged:

Expression 1b the mode of allowing the meaning of experience to become manifest. It requires the inquirer to partake deeply of the experience, rather than stand back in order to analyse, (Reason A Hawkins, 1988, p. 80)

The expression of experience includes its associated feelings, its passion, its past and future. With this emotive aspect, stories convey more than explanations can, and have a rightful, if heretofore ignored, place in science. Ian Mitroff described the inquiry style of the conceptual humanist who takes storytelling very seriously:

The best stories are those which stir people's minds, hearts, and s o u Ib and by doing so give them new insights into themselves, their problems, and their human condition. The challenge is to develop a human science that more fully serves this aim. The question then is not, 'Is storytelling science* but 'Can science learn to tell good stories?', (cited in Reason & HawkinB, 1988, p. 83)

The advocates of storytelling as inquiry do not claim that any story, is good science nor that stories can always stand by themselves. Instead, Reason and Hawkins describe a dialectic in which expressions illuminate explanations and explanations clarify expressions.

In the study at hand, there is an attempt at both explanation and expression. As noted earlier, the findings are organized and expressed in the form of three stories.

Within these storieB there are passages of extensive 108 quotations in which the respondents own words and own organization of their words is allowed to carry the research story, that is, with minimal intrusion from the researcher*s words. This is especially true for Story 3 in which the participants are grappling with the issue of how they are being held "accountable" by the state and the local community. Their efforts to articulate what is wrong with the current form of accountability and the professional dilemma it creates for them generate powerful and passionate expressions. Although I have selected and * arranged the quotations, as much as possible, the narrative attempts to let Foster participants tell this story for themselves.

In effect, the study's findings are presented as a combination of explanation and expression. Although my role as "explainer" is perhaps the more dominate one, there are occasional passages in which the participants* expressions take over.

Person of the Researcher

One of the axioms of the interpretive research paradigm is that it is not possible to fully separate the knower from the known (Lincoln & Guba, 1985). Any "external" world is perceived and filtered through the cognitive structures and values, and as the Sapir-Whorf hypothesis contends, language and culture, of the perceiver. Claiming that 109 science can be "objective", in the sense of operating independently from the scientist, is therefore impossible.

Given the acknowledgment that data are "constructed" or

"produced" by the researcher, rather than "found" (Simon &

Dippo, 1986), one of the essential aspects of both interpretive and critical research is to fully disclose the values and perspective of the researcher in order for the reader to be apprised of the impact of the knower on the known in the inquiry (Gitlin, Siegel, & Boru, 1988; Lincoln

& Guba, 1985; Mooney, 1975; Peshkin, 1988, Smith, 1988).

Forgiving the lack of inclusive language in his article on "The Researcher Himself", first written in the 1950s, one can enjoy the imagery with which Ross Mooney embraces the person within the researcher role:

Man is unable to leave himself out of his life and and, even though he may seem, in his own eyes to be temporarily successful, what happens in cases of self-denial is that the image denied entrance at the front door of consciousness enters unobtrusively by the back door of consciousness, clad in another garb, (cited in Pinar, 1975, p. 187)

With similar reasoning, Smith (1988) claims that

"relativism" is not a problem of consequence within interpretive approaches to research, but rather the

"inevitable consequence of our hermeneutical or interpretive mode of being in the world" (p. 18). In other words, once we come to understand that "objectivism" (a reality which exists "out there" independent of our 110 interest in it) is impossible, a relativism which acknowledges reason and rationality but only within a given conceptual scheme, paradigm, and context is not a problem for science. It is the inevitable way of knowing*

Rather than denying the values held by the person who is also the researcher, it is important to openly acknowledge and clarify them in the research process. In

Mooney * s words:

To be asked "not to be influenced by my values" is to be asked not to be influenced by my bonds of belonging or my tentacles of becoming. it is to ask the impossible, for what 1 am is involved in these. Values signify the inescapable necessity of man that he select some things from all other things for his appropriate use at each specific time and place. ...The more sensitive I can be in investing myself consciously in realizing my values through my research activity, the more profoundly I can penetrate universality. The road to conquest of sentiment and superficial identification is not through denial of sentiment and identification, but through its ripening into conscious valuing, (cited in Pinar, 1975, p. 192-3)

The open acknowledgment of what 1b being valued plays out in the ongoing methods of the research process. The practice of "participant observation", for example, has been attempted by researchers as if it were possible to observe without participating, or, as Qitlin et al. describe it, "as television watching", in which the researcher:

...watches the working of the Bchool and reportB his findings as if they were natural rather than constructed through the interaction of hiB theoretical perspective and the teachers1 work lives. (Gitlin, Siegel, & Boru, 1988, p. 19) Ill

Since it is not possible to observe without participating in some passive or active form, it is better for the researcher to acknowledge the effect of his/her presence on the observed, rather than to pretend one is "only watching". Furthermore, the traditional fieldwork caution against "going native", that is, getting bo caught up in the insider's perspective that one loses "objectivity", is replaced by Gitlin and his colleagues with a caution against "detachment":

The danger, for us, is not "going native" but detachment. The question is not whether the data are biased; the question is whose interests are served by the bias. As we leave behind the goal of objective knowledge, we are forced to consider the role social relations play in the construction of knowledge. (Gitlin, Siegel, & Boru, 1988, pp. 17-181

The social relations of the researcher and the researched are of concern both for how knowledge is constructed and how researchers and practitioners come to know each other as human beings. Rather than stay detached,

Gitlin et al . suggest that we enter the dialogue with the participants to create a dialogical community in which the researcher joinB in the work of transforming the context being studied (1988).

Reason and Rowan (1981a) emphasize the importance of the "authenticity of the researcher, and the need for the researcher to be involved as a whole person, not hiding behind a role" (p. xvii). Joining in on the messy 112 complexity of the lives we are trying to understand leaves researchers no choice but to reveal who we are as persons, our values, our strengths, our weaknesses, our histories, our personalities, our certainties and our uncertainties.

Not only are these disclosures important for establishing authentic human relationships, they are also the beginnings of the "reciprocity" of research which Lather referred to in her description of research as praxis (1986). Authentic understandings include authentic Bocial relations between researcher and researched.

Peshkin (1988) maintained that it is not good enough to say we as researchers are subjective, but it also is necessary to systematically seek out one’s subjectivity during the course of the study and describes how it is shaping the inquiry. He describes in some detail how his six different "I's" responded to what he was seeing and how they fought against each other throughout his year-long fieldwork in a high school.

In an attempt to follow Peshkin*s example, I have recognized the following "I's" in my year-long experience in the Foster schools. Over the course of conducting the study I became aware of (and at times noted in my methodological journal) that I was trying to be a number of different people/researchers at once. These different people included a: 113

- see what is happening I;

- support the beleaguered educator I;

- be liked and respected I;

- don’t be an exploitative researcher I; and a

- help them articulate an alternative response to the testing policy I .

Each of these "I’s" was present during different occasions

in the ongoing study. At the outset of the study I was most concerned with uncovering their interpretations of the policy. My focus was on listening to their words and watching their interactions.

As the study unfolded, and they articulated their reasons for being concerned about the possible ill effects of the state testing policy, I became concerned the policy could do more harm than good, at least in the context of this particular school district with its current faculty and approach to instruction. As described in Story 2 in the next chapter, my growing concern that these educators not be harmed by the misuse of the test results evoked my involvement in their preparation of the presentation of the test results to the school board and influenced me to urge them to consider alternative ways to be accountable for the education of the students, alternatives that would be more aligned with their professional identity and practice within Foster. This involvement represented not only my current task of being a researcher but also my past history 114

as a program evaluator, test developer, and consultant to

educators engaged in initiating new programs in schools. It

also reflects who I am as a person, and my perpetual

inability to remain detached from the circumstances, tasks,

and people around me.

In this regard, the current task of doing research haB

taken on a special meaning in my life. Mooney reminded us

that when research is allowed to come from within one's own

inner source of life, it becomes a way of generating and

serving that life. Reason and Rowan (1981a) insisted that

"research doesn't have to be another brick in the wall"

(the exact words used to describe it in a popular research methods course at The Ohio State University), but instead

can be something worthwhile for the researcher and for the

other people involved in it (p. xxiii). But research can also can become all-consuming, especially that done for a dissertation. As a result, I find the following quote from

the cover of Reason L Rowan's 1981 book to be very

compelling:

Who was that research I saw you with last night? That was no research That was my life!

Conduct of the Inquiry

This section begins with an overview of the methods used to complete the case study in the Foster schools during the period of January 1989 through March 1990. After describing how access was gained in the district, the specific techniques used to become familiar with the local context and to uncover how the educators, particularly the administrators, were making sense of the new policy are presented. Given that the person of the researcher is the primary instrument in naturalistic inquiry, the methods used to establish the trustworthiness of the study are reported. Particular attention is given to the occasions in which I was most actively involved in the work of the district. The approach to analysis of the vast amounts of data collected through qualitative methods is somewhat dependent on the characteristics of the data and the conceptual and organizational idiosyncracies of the researcher. For this reason, I have included a detailed description of my analysis strategies. The chapter concludes with a discussion of ethics, reflexivity, and the limits of the study.

Overview of the Study

Table 1 summarizes the formal data collection occasions and activities completed during the 15 months in which

Foster educators prepared for the implementation of both aspects of the state testing policy, the achievement and ability testing of students in grades 4, 6, and 8, and the 116

proficiency testing of students in grades 9 and 12. In addition to the 29 interviews, 37 observations, 13 sessions with the key informant, and eight member checks noted on

Table 1, there were numerous phone calls, informal interactions, observations, and conversations taking place before and after formal sessions, and while juBt "hanging out" in the school district.

Although I was in Foster over a 15 month period, the amount of onsite activity varied during the course of the study. As noted earlier, Foster was just beginning to prepare itself for the proficiency test which the state would first administer in fall of 1990, 20 months from the starting date of this study. Although educators were concerned about the proficiency test, at the beginning of the study its initiation seemed far away. On the other hand, Foster administrators had decided to prepare themselves for the 1989-90 mandate to administer achievement and ability tests to grades 4, 6, and 8 by giving the tests at these grade levels one year earlier than required, specifically late April, 1989. In this way they could develop the capacity to score and analyze the results internally {rather than send them to the test publisher). 117

Table 1. Data Collection Occasiona and Activities

Activities PhaBe 1* Phase 2** Total

Interviews (29)

Within Foster: (26) Central Office 2 2 4 Principals/ Asst. Principals 6 6 12 Teachers/Counselors 7 3 10 State: (3) Legislators 2 2 Department of Education 1 1

Observations (37) Meetings: (26) Regarding testing 10 13 Curriculum revision 9 9 School Board 1 3 4

Other occasions: 6 11

Reflection meetings 13 (with key informant)

Member Checks Date: Item: Given to: Feedback from: 2/89 Meeting description 1 1 5/89 Themes 1 & 2 15 7 9/89 Theme 5 (draft) 3 3 12/89 Theme 6 (draft) 1 1 2/90 Context (draft) 5 5 2/90 Context, Themes 1-6 27 5 3/90 Chapter IV (draft) 1 1

Documents Reviewed Pertaining to testing policy/procedureB: 19 Pertaining to overall context of district: 100+

* January - April 1989 ** May 1989 - March 1990 118

Consequently, the months of February, March and April

1989 were very active as teachers attended testing inservices and read instruction manuals, and administrators and support staff organized materials, schedules, and students for the testing. While this preparation was going on, I shadowed the assistant superintendent as he led meetings and inservice sessions and talked with individuals about the logistical and substantive issues surrounding the testing. In addition to observing these district-wide inservice sessions and meetings, I conducted my first

"phase" of interviews. I focused my initial attention on the administrators and teachers in the two middle schools.

In these buildings both the sixth and eighth grade students were being tested on achievement and ability and the administrators and eighth grade teachers were beginning to realize the implications of the statewide ninth grade proficiency test on the eighth grade curriculum.

In May, 1989 I prepared a written summary of the ideas

I had heard discussed in meetings and raised in interviews.

I identified six topics or "themes" and sent the list of themes and a seven page description of the first two themes to the 15 administrators and teachers interviewed during the first phase.

Phase 2 extended from May 1989 to March 1990. It included the end of the school year when educators were dealing with the test results, the summer months when the 119 year-long language arts/reading curriculum revision process began, and the fall and winter months of the school year which included the opening of two new schools and school board elections in the district. From May through early fall I completed interviews with the ten building principals, middle school assistant principals, and a sample of teachers in the middle schools, and continued to observe district-wide and individual school meetings in which the testing policy was discussed directly or indirectly as it pertained to curriculum revisions.

Starting in September, 1989 findings from the Phase 2 interviews were incorporated into the themes identified in

Phase 1 and early drafts of the new summaries were given to the key informant and others for initial feedback. At this point in the research process, my attention shifted to analysis and writing and less to data collection, although

I continued to attend occasional group sessions and kept meeting with my key informant to stay abreast of happenings in the district.

In February, 1990 a general description of the overall context of the district was written to introduce a 45-page document summarizing the six themes. This document was given to all 26 Foster educators interviewed as well as to the key informant. Revisions were made on the basis of their feedback and a shortened version of the themes is included as one portion of Story 1 in Chapter IV. Stories 2 120 and 3, summarizing both the effect of the research process on the Foster participants and myself, and the attempt to use the research as a springboard for new conversations regarding accountability, were drafted in February and

March 1990 and given to the key informant for review. His feedback on the draft of Chapter IV concluded the data collection process.

Gaining Access and Building Trust

Access to the Foster school district was obtained through contact with the assistant superintendent to whom I introduced myself after learning about him from a professor at the university. "Bill" had completed a qualitative research dissertation at the same university ten years before and was known to be interested in the effects of state policy on the teaching/learning process in schools.

As the Assistant Superintendent for Curriculum and

Instruction, Bill served as the primary district intermediary between policies made at the state level and the principals and teachers in the schools. With responsibility for overseeing the teaching/learning process he was in a position to know all the principals and teachers in the ten schools in the district, including the new ones hired each year to respond to the dramatic growth in pupil enrollment. In this capacity Bill provided introductions for me to principals and high Bchool 121

department heads, who in turn introduced me to teachers. He

also served as my "key informant" for the study (Bogdan it

Biklen, 1982). Lincoln & Guba (1985) described the key

informant as one who can provide "an inside view of the

norms, attitudes, constructions, processes, and culture

that characterize the local setting" (p. 258). Bill gave me

his view from the position of an assistant superintendent.

Given his "elite" role in the district, he could not

directly represent the perspectives of teachers and

principals.

Although Bill provided formal access to the district,

more important to the conduct of the research was obtaining

the cooperation of the other participants whose

perspectives were crucial to the study. Gaining the trust

of some of the administrators and teachers in this district

was the result of prolonged engagement (being seen in and

around a variety of meetings in the different buildings for

over a year1b time), being consistent and honest in

describing the purpose of the study, and scrupulously

guarding the confidentiality of the remarks made by one

person while talking to another (Bogdan &. Biklen, 1982;

Lincoln & Guba, 1985). The trust building process was assisted by my previous experience as a classroom teacher

and program evaluator, which contributed to my knowledge of

the language and flow of daily life in schools (Wolcott,

1973). 122

Appreciating Context: Document Review and Observations

To appreciate the context in which this particular state policy policy was being implemented, over 100 documents pertaining to the school district were reviewed.

These provided a sense of the history of the district, its current demographic status, and the pertinent issues affecting its present and future. For example, one of the most pressing situational factors impinging on daily life in Foster is its rapid growth over the past ten years and the projections for continued growth. The way in which administrators dealt with a new state policy mandate while they continually opened new buildings, recruited, hired, and reassigned staff, and enrolled and placed new students was one of the contextual issues affecting the implementation of this particular policy. Local documents reviewed included historical records, annual reports, school newsletters, newspaper accounts, curriculum guides, and sample notices to staff, parents and students.

Informal observations of the surrounding community, e.g., its stores, post office, and restaurants, augmented more formal observations of school board meetings, evening school-community functions, in-school assemblies, and social gatherings. These occasions provided detail to my perspective of the culture of the Bchool district. Special attention was given throughout the study to how the observed teachers and administrators interacted in terms of 123 both formal roles and informal relationships. In effect, my research attention was devoted to uncovering the organizational norms, the "how we do things here" among some of the educators in the district.

Making Sense of the State Testing Policy

In order to discern how Foster educators, especially administrators, were individually and collectively making sense of the state testing policy I used research methods which brought me in contact with their expressions about the policy. As argued throughout the previous sections of thiB document, it is through discourse that organizational reality is created and sustained and it is through the use of language that individuals make sense of and find meaning in their work lives. For the researcher, following the implementation of the policy thus means following and creating occasions for conversation about the policy. A detailed description of the different conversational settings is provided within the three stories of Chapter

IV. This section provides a general description of the research methods used, including participant observation, interviews, review of documents pertaining to the testing policy, and reflection meetings with the key informant.

Participant observations. Participant observations were used during meetings between and among administrators and teachers as the state testing policy was discussed. As 124

noted on Table 1, over the course of the study there were a

total of 13 meetings specifically called to attend to one

or more aspects of the state testing policy. These sessions

included district-wide meetings as well as thoBe in

individual schools. In addition, I attended nine meetings

of a year-long language arts/reading curriculum rewriting

effort, in order to sample how their deliberations were

affected by the state policy. Because the 19 member

curriculum revision committee consisted of teachers from all grade levels and all schools, as well as principals and central office administrators, it provided a rich setting

in which I could observe the deliberations and interactions among professional educators in the district.

Within the continuum of emphasis on participation on the one hand, and emphasis on observation on the other, the most suitable research role during these meetings was the one chosen by Wolcott:

"Participant as observer", a role in which the observer is known to all and is present in the system as a scientific observer, participating by * his presence but at the same time usually allowed to do what observers do rather than expected to perform as others perform. (1973, p. 7-8)

The focus for the observation at these meetings was on recording the conversations, especially direct quotations regarding the state policy, with the notation of who speaks and who raises which issues. Most of these notes were taken 125

by hand, although on a few occasions the sessions were

audiotaped. In the initial meetings with 15 to 20

participants, I felt that asking to tape record was an

inappropriate request from someone who had just been

introduced as a "guest" into an ongoing social group.

Eventually, as I became a more common part of such meetings I aBked and received permission to record them. I

found that the recordings were not always more useful than

the detailed notes I could write, since it was difficult to distinguish among many voices on a tape. When the audiorecordings were made, they supplemented, rather than replaced detailed hand recordings of the conversations. As described in more detail in Chapter IV, although my tendency was to remain quiet at these meetings and focus on note taking, if I had a point of information to contribute to the discussion, or if I was asked a direct question, I freely participated in the conversation.

In addition to focusing on the discourse of participants at meetings, I attended to the relationships and interactions between and among teachers and administrators as they prepared to implement this policy.

Evidence of the relationships was available through both their verbal and nonverbal communication patterns, for example, how they greeted and took leave of one another, how they referred to each other in the presence of the researcher; how they oriented their bodies to each other 126

while speaking and listening. These nonverbal expressions

provided some definition of the relationships the educators

understood to exist between and among themselves (Galloway,

1977; 1984; Goffman, 1971; Hall, 1974; 1976;

Rosenstock-Huessy, 1970). The effect of the state testing

policy on the professional relationships among educators

was one of my a priori and ongoing research questions. I

therefore consciously watched for signals which

characterized their relationships during the many formal

and informal observation occasions within Foster.

To the extent possible, I attempted to "shadow" Bill as

he carried out his assistant superintendent

responsibilities regarding the state testing policy.

Although shadowing was possible during meetings and

discussions planned ahead of time, it was not possible for

the many unplanned asides and serendipitous conversations

in which the policy came up as tangential to other

discussions.

Interviews. Extensive data were collected from Foster

administrators and some teachers during 26 unstructured

interviews. In addition, three interviews were conducted

at the state level to ascertain policy makers intentions with regard to the testing policy. The two state

legislators most responsible for authoring the 1987 Ohio

law and the Ohio Department of Education director most

closely associated with the administrative rules and 127 regulations supporting the legislation were interviewed during the study.

The overall question for interviews within Foster was how, if at all, would the state testing policy influence their work and relationships in the district. Although I would occasionally aBk a clarifying question or would ask the respondent to elaborate on a point, I did not have a formal list of questions to guide the interviews, especially during Phase 1 (January to April, 1989). My intent during these sessions was to tap into their experience, to allow them to teach me what was important about the testing policy. These interviews served to access their meaning regarding this new policy, its dilemmas, demands and expectations.

As Lincoln and Guba (1985) claimed "the search for meaning is a search for multiple realities, truthB and perceptions" (p. 157). The multiple truths, perceptions, and realities surrounding the implementation of this newly mandated state policy in Foster were uncovered by asking these educators how they each described the policy and the organizational context in which the policy was entering. An attempt was made not to impose my view of the policy on them, but instead to respect their way of constructing its meaning for their own work lives. The hope was to let them speak in their own voice, as Mishler (1986) described, in the "voice of their lifeworld, that is, individuals' 128

contextual understanding of their problems in their own

terms" (p. 143).

The interviews typically lasted about an hour, although

a few were thirty minutes and many approached two hours.

Twelve building administrators were interviewed: the

principals of the seven elementary, two middle schools and

high school, and the assistant principals of the two middle

schools. In addition, four central office administrators

and support staff who report to the Assistant

Superintendent for Curriculum and Instruction were

interviewed. A total of 10 middle school teachers and

counselors were formally interviewed. (Conversations with

additional teachers from the high school and elementary

schools were held as part of observing department meetings

and curriculum revision meetings.)

The assistant principals at the middle schools arranged

for me to meet and interview the teachers there. Teachers

were chosen on the basis of teaching a grade level (6 or 8)

and subject matter (English, reading, writing, mathematics,

and social studies) included in the achievement and/or

proficiency tests, as well as their availability at the

times I was in the building. This process was a rather haphazard and serendipitous one. Although I was able to

schedule some appointments ahead of time, it generally worked best for the administrators to have me come into the building for several hours each day and let them see which 129

teachers they could find who were free (and willing) to

talk to me. Coupled with the fact that I was able to talk

to other teachers at meetings and that administrators were

told that I wanted a variety of opinions, I accepted this

sampling method as reasonable for my purposes and as

indicative of and authentic to life in schools.

As noted earlier and described in more detail in Story

2, the interviews were started with an open ended question

about the effect of the policy on their work and

relationships. In this manner each respondents raised some

or all of the issues summarized at the end of Phase 1 as

the six "themes" of the findings: 1) the use of the test

results; 2) the effect of the testing on relationships; 3) whether the achievement testing should be done in fall or

spring; 4) the effect of the testing on the local

curriculum; 5) its effect on local resources; and 6) how

testing fit into what it meant to be accountable to the

local community and the state.

During the second phase of the interviews I was

interested in verifying whether these six themes would continue to be the most Balient issues regarding the policy. If, toward the end of.the interview the respondent had not yet addressed one or more of the themes during the course of our conversation, I would solicit his or her opinion about the issue. In general, respondents would connect their previously expressed ideas to my identified 130 theme. For example, if the respondent had been talking about competition, and I then asked about the effect of the testing policy on relationships, he or Bhe would add how competition affected relationships in a school or district.

In this way I amplified and verified my findings regarding the six themes, but remained open for additional issues to be raised by Phase 2 respondents.

Respondents for both Phase 1 and Phase 2 interviews were given a one-page handout describing my study {See

Appendix A) and were asked permission to tape record the sessions. Although handwritten notes were also taken, the audiotapes were helpful to insure the accuracy of quotations. The recording of precise wording was especially important for remarks directly related to the respondent’s view of the state testing policy. Representative examples of extensive quotations are included in Chapter IV.

Document review. As Foster prepared to implement the state testing policy a "trail" of memos, planning guides, test instructions, test score analyses and summaries was generated. Nineteen of these documents were reviewed for the insights they offered on how the policy was being characterized in the district, how administrators and teachers were incorporating it into their existing responsibilities, and the additional demands it was making on their time, energy and attention. The documents also served as a tangible summary of the logistical tasks needed 131

to accomplish the testing itself as well as any changes in

curriculum and instruction engendered by the testing

policy.

Reflection meetings. As the intermediary between the

state policy makers and the administrators and teachers in

schools, the assistant superintendent’s perspective was

crucial to the district's response to the state testing

policy. For this reason, I scheduled periodic "reflection

meetings" with Bill. These sessions served several

purposes. They allowed me to catch up on events and

conversations regarding the policy that had occurred

outside of formal sessions or Bince my last visit to the

district; they allowed me to gain a detailed view of Bill’s

sense-making about the policy and curriculum and

instruction in general; and they allowed me to bring back

my constructions of findings early and often in the

research process.

Over the 15 months of the study, Bill and I held 13 meetings specifically for the purpose of reflecting on the

issues raised by the research. I would generally lead off

the meeting by asking if he had issues to raise. If so, I

tried to attend to those first. I would then bring up my

questions or comments based on the data collected and analyzed Bince our previous meeting. In response to my questions Bill would elaborate on the history of the district, or provide details about the present context, or 132 describe his own views about education and organizational change. As reported in Chapter IV, these conversations allowed us to get to know and trust each other, and resulted in Bill allowing me wider and deeper access to the district and its members.

Establishing Trustworthiness

In addition to the research activities focused on the sense-making of Foster participants, the methodology included strategies designed to enhance the trustworthiness of the findings. As discussed below, I conducted member checks, kept a methodological journal, and met regularly with a peer debriefer. In addition, I triangulated the findings by combining the methods of observation, interviews and document reviews, and by verifying ideas raised by one person with several others and with my own observations.

I also sought out disconfirming evidence. For example, when a group conversation among principals seem to indicate that one of them was a strong defender of the need for standardized achievement testing, I sought out this person for my next interview. I assumed (hoped?) that this principal would give me a different point of view from the others I had interviewed to date. As it turned out, although the potential merits of the testing were articulated, thiB person was one of the most passionate 133

objectors to the way in which the state testing policy was

being used to set schools and teachers in competition with

each other and to narrow the curriculum to the isolated

facts and skills that were measurable with these typeB of

tests. As the study progressed) and the most common opinion

expressed was one of concern over the potential ill effects

of the testing policy, I continued to search for a strong

proponent of the policy, especially among administrators. I

never found one. I spoke to a few individuals who felt

neutral about the policy, but most of those I interviewed

directly or listened to in meetings were either ambivalent

(could see some potential benefits and some potential harm)

or strongly opposed.

Member checks. My primary research focus was on

discovering what meaning Foster participants were making of

the state policy. As quoted earlier, Lincoln and Guba

(1985) characterized the search for meaning as "a search

for multiple realities, truths, and perceptions” (p. 157).

The credibility of a naturalistic study rests on its

authentic representation of the multiple realities

constructed by the participants. The question becomes whether the researcher*s reconstruction is an accurate

rendering of the original multiple realities constructed by

the participants. The most appropriate way to answer this

question is to check if the reconstruction is credible to

the original constructors (Lincoln & Guba, 1985). At 134

various points throughout the study I conducted "member

checks"; that is, I presented my interpretations of what I was seeing and hearing to Foster participants for their confirmation and disconfirmation.

Whenever possible, I carried out an on-the spot member check at the conclusion of each interview. I briefly

summarised what I felt were the most salient points made and asked the respondent to confirm or disconfirm my choices and tell me if I had described his or her views accurately. The more formal member checks occurred with the written documents, in which I summarized points of view across multiple observations, interviews and document reviews. As noted in Table 1, written member checks were done at seven points in the study with one or more participants asked to give feedback.

The first attempt at recording and sending back what I perceived to be going on occurred in the first weeks of the study when I attended a meeting with Bill and teachers at the high school. The teachers were confused about how to complete a form generated from Bill's office pertaining to the district’s preparation for the state proficiency test.

After the meeting I summarized what I felt were the main pointB of confusion as well as teachers' overall comments regarding the testing policy. As noted in Chapter IV, this memo to Bill was an important first Btep in the research process both because it provided him with useful, if 135 disconcerting, information and because he in turn allowed me wider access to people and issues in the district.

One of the two occasions in which all the parties to the research had a chance to respond to my description of their collective opinions was in May, 1989 when I sent a seven page single-spaced document listing the six themes I had identified and an elaborate discussion of the first two themes to each of^the 15 people interviewed to date. Seven people responded to this summarization of my first phase of the study; all of them confirmed its accuracy.

From September onward, subsequent drafts of the findings were given to Bill and selected others as I incorporated Phase 2 data into my summaries. For example, the initial draft of Theme 5, which pertained to the effect of the testing policy on the district’s resources, was shared with the three people who had provided me most of the specific information regarding costs and time spent on the various logistical tasks related to the testing. At another point, when I wanted to verify my description of the overall professional context for principals in the district, I took my written draft to five administrators and asked if they agreed with my characterization. The revised versions of these drafts were included in a 45 page single-spaced summary of Themes 1-6. In February, 1990 all

26 people interviewed and the key informant were asked for feedback on this presentation of their collective opinions. 136

As described in Chapter IV the process of doing member checks not only provides the researcher with confirming or disconfirming judgements on her interpretations, it provides the respondents an occasion to reflect on a picture of themselves drawn from an outsider's point of view. Sometimes the picture can further understanding or prompt a new action.

Methodological journal. I began my methodological journal in January, 1989 and recorded entries through April of 1990 (in two volumes). In addition to noting the decision points in the study (for example, the choice of

Foster as the site and the testing policy as the state policy of interest) and questions about the methodology

(for example, my pondering over how the principals were finding teachers for me to interview), it became a forum for articulating feelings, especially uncertainties and puzzlements (for example, how much of myself to reveal in the early interviews) and for trying out various interpretations of the data. I also recorded analysis decisions and strategies, code words used, and reflections on writings that were important to my conceptualization of policy, language, organizations, and critical ethnography.

The journal entries proved useful as an intellectual and emotional history of myself as researcher. They also provided me with "next steps" in the data collection process. As I prepared for subsequent interviews and 137 observations, the issues and feelings raised in the journal alerted me to ask about or watch for evidence of hunches or interpretations that I was formulating.

Peer debriefing. Lincoln and Guba recommend the technique of peer debriefing as a way to keep the researcher "honest" (through the searching questions of a devil's advocate); to test out emerging working hypotheses; to consider next steps in the unfolding design; and to provide catharsis for the confusing emotions of the lonely inquirer (1985, p. 308). My peer debriefer served all of these functions in addition to being a sounding board for ethical dilemmas raised during the inquiry. As a fellow graduate student in education, she knew enough about schools, teachers, and administrators to understand the context of my study, but her research, although qualitative, was focused in a different area of the field.

As a result I needed to clarify my own thinking about policy, culture, discourse, organizations and the involvement of the researcher in the context in order to explain myself to her. We have held 13 sessions to date, of two to four hours in length. Within each session we discussed aspects of research design, methods, analysis, inferences, and approaches to summarizing the findings.

Researcher involvement. Researchers in the critical paradigm argue for the involvement of the inquirer in the dialogue and work of the participants being studied, as well ae for the disclosure of the self of the researcher as

a first step in establishing reciprocity in the researcher

- researched relationship (AnderBon, 1990; Carr & Kemmis,

1986; Gitlin, Siegel, & Boru, 1988; Heron, 1981; Lather,

1986; Reason & Rowan, 1981a). As I came to know and became known to many of the administrators and some of the

teachers in this district, I became more involved in their

concerns. As noted above, my subjectivity in the research

took the form of several "I's". I wanted to find out what was happening in their minds, yet not exploit them for my

research purposes; I wanted to support their efforts at establishing and maintaining a professional culture and not

see them harmed by misuses of inappropriate test data; I wanted to be liked and respected by them, and, as the study unfolded, I wanted to help them articulate an alternative, more appropriate means of being accountable to an increasingly skeptical community.

The phases of the evolving researcher-researched relationship are described in some detail in Story 2 of

Chapter IV. At this point, it is important to highlight the form my involvement took. The first and most dramatic activity was in helping central office administrators understand technical aspects of the achievement test data they had scored and were preparing to analyze. My past work in testing and program evaluation proved useful in explaining the properties of different types of test 139 scores, and in considering how Foster administrators might present summaries of their students1 Bcores to the local school board. In essence, I helped in two ways: discussing technical aspects of the test characteristics and deliberating on how to present results from different grade levels, subject areas, and school years in an intelligible manner to a lay audience, and in a way that avoided comparisons being made across schools.

After the presentation to the school board was completed, I was asked by the assistant superintendent to consider teaching a course on assessment in the district.

The idea was for me to join with faculty at a nearby university to present a year-long course to Foster teachers on understanding standardized achievement tests and improving classroom assessment practices. As the study progressed, and the need to articulate a more appropriate local accountability approach became evident, we added the development of an "expanded accountability statement" to the potential content of the course. Although we have had discussions about thiB possibility with faculty from the local university, to date the course has not materialized.

Although the topic was never raised by my key informant, other administrators in the district urged me to consider working in the district once I had completed my dissertation. While flattered by their affirmation, I found these conversations disconcerting. As described in Story 2, 140

I did not want concern over future employment possibilities

to cloud my mind and emotions while I was engaged in this

case study.

The final aspect of my involvement, as recounted in

Story 3, was my attempt to use the research process, and

especially the member checks, as a springboard for Foster

administrators to start the collective conversations needed

to develop an alternative means of being accountable for

student learning. Given that they disliked and distrusted

the means chosen by the state testing policy, and they

lamented their lack of being able to articulate their vision of the educational process and how it might be authentically measured, I hoped that the knowledge constructed in the research process would serve to activate them to transform the situation in which they found themselves.

Balancing experience and reflection. The research process I used in Foster was characterized by a back and forth movement between involvement on site and time to reflect about the experience. Heron (1988) argued that validity in cooperative inquiry is partially acquired by balancing experience and reflection, by not prolonging either before moving back to the other way of knowing. I immersed myself in Foster's preparations for implementing the state testing policy during the first phase of data collection from January to April 1989, and then pulled back 141

to write up my findings in early May. Then I took this writing back for confirmation before I began another round of interviews and observations in late May and early June.

I repeated this cycling through the second phase of the study and as a result I was able to constantly construct, reflect, and check on my constructions throughout the 15 month research process. In combination with the strategies of triangulation, peer debriefing, keeping a methodological journal, and prolonged engagement in the field, I believe the cycling between onsite experience and offsite reflection and analysis served to confirm that the findings reported in Chapter IV are not figments of my imagination.

Data Analysis Process

The particular strategies used to organize and analyze the bits of data from different sources into the three stories in Chapter IV are presented below. This description includes an explanation of the unfolding conceptualization of the study because insights into interpretation and presentation also influenced subsequent data collection and analysis. The following account presents my actions and emerging thoughts as they occurred chronologically in the course of the inquiry. 142

Phase 1: (January to April 1989)

Although transcripts and field notes were read through as they were prepared, the first concentrated and systematic analysis work began in May, 1989 using what was then dubbed "Phase 1 data", that is, data collected from the start of the study in January through late April. All of the interview transcriptions, field notes from observations, and documents available at that time were read through several times until six major topics (later called 'themes') Beemed to emerge (obviously, from my perspective). As noted above I organized the remarks made about the state testing policy into the following six main themes:

1. How the test results would be used locally and statewide.

2. The effect of the state mandated testing on the relationships among educators, between educators and students, and between educators and the local community.

3. The pros and cons of fall versus spring achievement testing.

4. The effect of the state mandated testing on the local curriculum.

5. The effect of the state mandated testing on local educational resources (money, time, energy and attention).

6. How the state mandated testing fits into what it means to be accountable to the local community and to the state. 143

Themes 2 and 4 (effect of the state testing policy on relationships and local curriculum) were a priori ideas with which I entered the research site. Themes 1 and 3 (use of the tests and fall vs. spring testing) were identified during the data collection period. These four topics were presented to the key informant during a "reflection meeting" in late April as findings to date. The final two themes (use of local resources and what the state policy means for local and statewide accountability) were identified during the systematic analysis process in May,

1989.

In addition, there were several ideas related to the research process itself that were identified but not analyzed at this Btage:

- the use of the construct "policy and conversation" as the rationale for focusing on how people made sense of the newly mandated state testing policy;

- issues of ownership of data (e.g., in whose 'voice’ would the interpretations of the respondents be given);

- reactivity; does my asking about the state testing policy put more attention on it than local educators would have otherwise given? what effect does my doing the research have on the people in the district?

These three issues became the focus of more concentrated data collection and analysis efforts during the second phase of the study.

The six substantive topics were each assigned a section in an 8.5 x 11 "analysis" notebook. The analysis process 144 was essentially one of going through each of the transcripts and field notes and recording each distinct idea into one of the six segments of the notebook. The ideas were summarized into short, cryptic phrases. In the margin next to the phrase , I would record the source of the idea. When more than one person, or more than one set of field notes made the same point, I would note all of the sources for that point in the margin.

When finished with this process I had from two to seven handwritten pages of distinct ideas for each theme. The next step was to organize these ideas into sets of related ideas and put these sets into a reasonable sequence in order to write about them. For example, under Theme 1 ("use of the test results"), ideas coalesced around the subtopiCB of positive and negative uses; using test scores to compare teachers, schools, and districts; using tests for political, not educational, purposes; using tests as if they give more information about the student or the learning process than they do, etc.

These subtopics became the focus of paragraphs in the written summaries of the findings from this phase. As ideas were included in the written narrative, they were "checked off" in the notebook. The written description continued until all ideas recorded in the notebook under a given theme were presented in writing. 145

The first two themes were summarized in this way

(approximately three single spaced pages each) and were used for the first member check the 15 people interviewed during Phase 1.

Phase 2: (May 1989 to March 1990)

As described above, Phase 2 included the end of the

1988-89 school year, the summer months and through March of the 1989-90 school year. During this time, additional administrators and teachers were interviewed and observations of a test administration and meetings were conducted. At the same time, I was drafting the narrative summary of Themes 3, 4, and 5 using the Phase 1 data analyzed and organized via the notebook noted above. I sporadically added additional data from the ongoing observations and interviews, especially if new information contradicted that included in the Phase 1 data.

At the end of the summer, the Theme 5 summary (the use of local resources) was finished in a form available for a member check (seven single spaced pages). In September,

1989 it was given for feedback to the key informant and two other members of the central office staff who had provided me most of the information on the funds and time expended implementing the state testing policy. Themes 3 and 4 (fall vs. spring testing and the effect of state testing policy on curriculum) remained in draft form until the additional 146 data from Phase 2 was systematically incorporated into the analysis and writing.

By late September 1 had gone through several versions of how to think about the various "layers" of this study and, with the help of one of the co-chairs on my dissertation committee, had settled on the notion of organizing the data, the analysis, and the writing around three "stories": 1) a straightforward description of the how the state testing policy affected the organizational context of the school district and how individual administrators reacted to and made sense of the policy;

(which incorporated Themes 1-5 first identified in the

Phase 1 data); 2) an account of the effect of doing the research on the people in the school district and on myself as researcher; and 3) an account of the effects of combining the state testing policy itself and my research within the organizational context of this district. This last story has the added dimension of my trying to influence their response to the state policy by asking them to deal more directly with the issue of what it means to be accountable locally and to the Btate (Theme 6 from the

Phase 1 data).

Running through all three stories was the focus on

"policy and conversation", by which I meant listening to how participants spoke about the state testing policy as a way of learning their personal and collective sense making 147 about it and response to it and considering how the discourse rules of the context shaped their conversations about the policy.

The organization and analysis of the Phase 2 data were influenced by the conceptual picture of the three stories and the "policy and conversation" strand running through them. In September, I made the decision to not transcribe every word spoken on the taped interviews, but instead to use the notes taken during the interviews as a guide to those sections of the tape that included especially clear, coherent, and impassioned renderings of the person’s views about the testing policy. The transcribed portions of the tape were subsequently entered into a qualitative data analysis computer program called Ethnotfraph. The remaining ideas from the interviews and the field notes were recorded in the analysis notebook as before, using new sections for the original six themes as well as for other topics important to Stories 2 and 3 (e.g., feedback from member checks; reactivity; organizational context, etc.)

Portions (and sometimes whole conversations) from Phase

2 interviews were put into Ethnograph and coded with 19 different code words (six for the first six themes and 13 additional ones related to how the research was conducted; the organizational context; and the nature of their discourse). Ethnograph stores these codes and makes the segments accessible for analysis by one or more code words 148 at a time. For example, the code word 'accountability’ was assigned to all interview and field note segments dealing with the issue of how the state testing policy impacted on what it means to be accountable to the local community and to the state. On command, the computer Borts through all the transcripts and field notes and provides a printout of just those segments that pertain to respondents* views on accountability.

To summarize, Phase 2 data were treated in two ways.

All ideas were put into the handwritten "analysis" notebook under one of the original six topics or additional emerging topics, and/or were transcribed and coded into Ethnograph for computer sorting. The decision of what was put into

Ethnograph versus what was paraphrased into summary statements for the notebook was based on my searching for people’s own phrasing of their interpretation of the testing policy. I wanted their views in their words.

Direct, clear, coherent, and impassioned statements were put into Ethnograph so they would be available for quotation in my written summaries of the findings. In keeping with the focus on conversation, I wanted their words to convey their feelings about the policy.

Many of the interviews were over an hour long and not everything discussed was related to the testing policy.

Some ideas were repeated several times within an interview, and not everything was stated in a clear, quotable way, so 149 not everything was transcribed and coded into Ethnograph.

But all ideas regarding the testing policy were recorded at least once, either into a particular segment of the handwritten analysis notebook or into the computerized, coded files. Some ideas were included in both places.

When I began writing about the accountability topic, it was for the express purpose of having Foster administrators see their collective view of this issue and to decide if they wanted to tackle a reframing of the issue. In effect, the overall finding was that participants were not happy with the state enforcing the use of standardized test scores as a measure of whether or not schools were successful, but educators have not articulated an alternative to this approach. As described in Chapter IV, my writing of the accountability theme findings was intended both to acquire their reaction to the accuracy of my account, and to suggest that they might want to act on this issue and develop an alternative approach to local accountability than the approach of test scores to which they were objecting.

In effect my audience for this portion of the findings was the Foster administrators, not the readers of this dissertation. In the hopes of spurring them to action, I developed a summary which included a list of quotes from the interviews on the left and a list of one to two sentence statements about the issue of accountability on 150 the right. My hope was that seeing their own words would be a more powerful rendering of the topic than my narrative description of their ideas.

The analysis process used to develop this written account included doing a computer sort of all interview segments coded for 'accountability', and reviewing the summary phrases recorded under this topic in the analysis notebook from both the Phase 1 and Phase 2 data sets. Using these three sources of data, I generated an eight page expanded outline of all the ideas (the sources for each idea were noted next to the item on the outline) and the outline was used to develop a seven page single spaced list of quotes on the left and summary statements on the right.

Quotes were chosen for inclusion on the basis of their being the most complete, clear, coherent and compelling statements of the idea. (Often exact quotes are not complete sentences with clear referents. I choose those that could best stand on their own and that expressed strong views of the respondent.)

Ethics/Reflexivity/Limits of the Study

The question of ethical conduct has always been of utmost importance in ethnographic investigations of the meanings others make of a phenomenon. In addition to the researcher's acquisition of an interpretive understanding

(verstehen) of those being observed, there iB also a 151

growing concern for reflexivity, that is, the uncovering

and revealing of how the researcher's biases, interests,

values and personal interpretations affect the inquiry

(Anderson, 1989; Lincoln & Guba, 1985; Simon & Dippo, 1986;

Wolcott, 1973). This section describes how the issues of

ethics and reflexivity were addressed in the study and

concludes with a discussion of the limits on the

interpretations of its findings.

Two immediate ethical issues pertain to any inquiry

using human beings: 1) the voluntary participation of

respondents who have been given enough information to

understand the nature of the research and its risks and

obligations; and 2) protection of respondents from risks

(Bogdan & Biklen, 1983; Erickson, 1986). In addition,

gaining access into a site for in-depth Btudy places

special constraints on the researcher. Protecting

participants from the risks associated with exposure of the

site is even more difficult when a detailed rendering of

the particular context is a significant part of the study.

As noted above, each person interviewed was given written information on the study and asked for his/her permission prior to conducting the interview. In addition,

the designated "leader" of a given meeting was asked beforehand if the researcher might attend. Furthermore, as part of the informed consent I alerted potential participants that although my intention was to maintain the 152

confidentiality of both the district and their personal

identities, strict anonymity could not be promised since it

was always possible for someone else to discover and reveal

the identity of the schools in such a study. The immediate

and constant use of pseudonyms, refraining from discussing

the research site with colleagues and others, keeping the

notes unavailable to anyone but myself, and disguising the

identity of the site in writing up notes and summaries were

precautions taken to protect the confidentiality of the

participants in this study (Wolcott, 1973).

The effect of having an outsider observe the ongoing

work lives of administrators and teachers was another

ethical issue for the study. As Wolcott (1973) noteed: "No

one, myself included, had anticipated all the situations in which there might be stresses in having an outsider present

as observer" (p. 4). Although I intended for the research

to be mutually beneficial, the unrelenting position of the

researcher as new to, and not of the setting, creates

additional demands on the observed. These demands included

explanations about "ordinary" events as well as the

exercise of even more caution in delicate situations. Being mindful of the additional demands my research made in the already stressed lives of Foster participants, yet at the

same time wanting to insure that my interpretations were accurate, and thus asking for more of their time for member checks and reflection meetings represented a constant 153 ethical and logistical dilemma for me during the course of this study.

Another ethical issue pertained to the ownership of the data. Given that the written report represents the researcher’s reconstructions of the participants’ multiple constructions of reality (Lincoln & Guba, 1985), and its credibility depends upon the participants accepting it as an accurate rendering of their interpretation, what happens to the research when one or more participants disagree with the portrayal? As Anderson & Kelley (1987) claimed, and

Lincoln & Guba (1985) suggested, negotiated outcomes are possible, and although not every negotiation will end in agreement, each participant has a right to have input about the final product and the researcher is obliged to honor that input. Ball (1983) and Erickson (1986) noted that the written report serves as a collective presentation of self by the participants in the study, and they may not be impressed with the researcher’s rendering of their group portrait.

I was fortunate that Foster participates affirmed my written representations of their collective perspectives.

Although there was an occasional suggestion for a minor wording change, and some people added dimensions to the account, of those who gave me feedback, no one objected to the overall accuracy of the written summaries. 154

As discussed previously, the primary instrument for the study has been the researcher herself (Bogdan & Biklen,

1983; Cusick, 1973; Gitlin, Siegel & Boru, 1988; Lincoln &

Guba, 1985; Peshkin, 1988; Smith, 1988; Wolcott, 1973),

Whether through participant observation, shadowing of the assistant superintendent, or noting the organizational and social arrangements of the school district, the data for the inquiry has been collected in the form of field notes and handwritten and/or audiotape transcriptions from interviews. As a result the biases, interests, values and perspective of the researcher has influenced the direction of the study as well as the data collected.

The concern for neutrality addressed in conventional social science inquiry is modified in interpretive and critical inquiry to address the explication of just how the researcher influenced the research. In other words, since the effect of the researcher on the research cannot be prevented, it is assumed and documented. The strategies used to document my influence included maintaining and organizing the raw data and the summaries and interpretations in such a way that an auditor can trace the data to the same or similar conclusions, keeping a reflexive journal (which chronicled personal feelings, values, interests, confusions, speculations on growing insights and alternative interpretations of the data), and meeting with a peer debriefer to heighten my awareness of 155 my values and biases as the data were being produced.

I also attempted to chronicle the reactive effects of

the research. Of particular concern was the effect of my

questioning of participants about behavior which had

previously been taken for granted. Wolcott (1973) argued that such questioning can result in more conscious behavior during the next occurrence. This focusing of research attention on a given topic is especially problematic when one of the researcher’s questions pertains to the amount of attention normally given to the topic. In the case of investigating the influence of the newly mandated state testing program on the work and relationships of Foster educators, I was very concerned that the very act of asking the question might focus more attention on the state policy than would have ordinarily occurred in the normal course of the year’s events in the school district.

To assess the extent to which my questioning about the policy affected their view of it, participants were asked about the influence of the research itself during the first member check of the findings in May 1989. The response from most who addressed this question was that the research helped them to clarify their thinking about the policy, but did not change their attitude toward it. As described in

Story 2 of Chapter IV, the assistant superintendent felt that my asking did encourage people to think about the impact of the testing policy on local curriculum and 156 instructional practices more than they might have otherwise. In this way the research question may have evoked more conscious attention to the overall effect of state testing policy on the everyday life in Foster schools.

Beyond the impact of the research question on the researched, the explication of the limits for a critical study must address how the social construction of reality described in the account first came to be constructed in that particular way (Anderson, 1989; Deetz & Kersten,

1983). In addition to noting how the historical, material and social conditions of the wider cultural context affected the sense making of the participants, such reflexive questioning asks about the ideological biases of the researcher and the social relations established between the researcher and the participants in the process of conducting the inquiry.

Earlier in this chapter I characterized this inquiry as a modest attempt at research as praxis in its efforts to promote participant self-understanding and encourage self-determination (at least with regard to the issue of accountability) (Lather, 1986). As a part of this limited effort to consciously impact the participants I need to be careful not to let my interest in activating their efforts become a "container into which the data are poured"

(Anderson, 1989, p. 254). Anderson noted five interlocking 157

dimensions to reflexivity for the critical researcher. I

have listed these below with an example of each dimension

from this study included in brackets:

a. the researcher’s constructs [e.g., the interaction of culture, language, organization, context and policy shaping members’ discourse]

b. the informants’ commonsense constructs [e.g., their sensemaking about testing policy before the research began]

c. the research data [e.g., participants’ written and spoken words to each other and to the researcher]

d. the researcher's ideological biases [e.g., belief in educators' professional culture rather than legislated learning as an effective strategy for school improvement]

e. the structural and historical forces that informed the social construction under study [e.g., the arrangement of financial wealth and organization of schooling in this country which provides suburban districts the resources to select teachers capable of creating and sustaining a professional culture] (1989, pp. 254-255).

Each of these dimensions interacts with the others to limit the claims of the study. The following attributes of the way in which this inquiry was conducted by this researcher place limits on the conclusions that can be drawn from the findings reported in the next chapter:

- the restricted focus on the conversations among educators (as opposed to, for example, conversations between educators and parents, board members, students, etc.). Within the educators, I focused especially on administrators, and within administrators I gave special attention to the meaning-making of the assistant superintendent who served as the key informant; 158

- the possibility of the researcher being co-opted by overly relying on the perspective of the key informant;

- the researcher’s coining to agree with the participants on the potential ill effects of the state testing policy on the educational practices of teachers and the educational outcomes for students;

- the researchers lack of attention to the historical and material influences that shaped the organizational arrangements and financial resources of schooling in this district and the effect of these forces on the social constructions of the participants.

With these limitations cautioning against over interpretation, our attention now turns to the findings of the study. CHAPTER IV

FINDINGS OF THE STUDY

Introduction

This study focused on the interaction of an external state policy, a school district organization, and the individual and collective perspectives of administrators and teachers. It examined how educators made sense of a state testing policy and how they incorporated it into their established beliefs and practices.

The research sought to uncover the perspectives of the educators about the state policy through their discourse.

By focusing on the conversations surrounding the state policy, both those occurring naturally in the school district and those initiated by the researcher via interviews, the study attempted to reveal the meanings local participants ascribed to the policy. In addition, the rules of discourse which shaped the talk about the policy were explored. Discourse rules included, for example, the constraints placed on the types of conversations allowed in a setting, the topics permitted on the conversational agenda, and the way in which participants relate to and address one another as they talk in a given context.

159 160

The research findings are presented in the form of three distinct yet related stories, each with a focus on conversation. The findings are organized sb stories to capture three different aspects of the inquiry as they unfolded somewhat simultaneously during the course of the study. By framing the results as narratives it is possible to focus on one "storyline" at at time, while recognizing that all three stories are interwoven and unfolding at about the same time through the actions of the participants and the researcher. As noted in Chapter 111, the storytelling framework also allows for the expressions of participants occasionally to predominate over the explanations offered by the researcher.

Story 1 traces the impact of the state testing policy on the professional culture of the educators, especially the administrators and their relationships with each other and teachers. It addresses the question: How does the state policy relate to the conversations in the professional culture? Educators’ initial reaction to the policy and their predictions for how it will influence their work and relationships are described. The rules for discourse which are implied in the policy are contrasted with the discourse rules inherent in the professional culture. The 161 implications of these differing rules for the conversations among educators and between educators and the public are then explored. The talk of administrators is followed most closely because they serve as intermediaries between the state policy and the teachers and students.

Story 2 focuses on the research itself and addresses the question: How does the research process relate to the individual conversations in the professional culture? After a discussion of the discourse rules implied in the research paradigm and methods, the effect of conducting the inquiry is described in terms of its impact on the conversations and actions of both the participants and the researcher.

Story 3 links the state policy and the research and asks the question: How do the policy and research combine to bring about new conversations? This story recounts how the researcher attempted to use the inquiry as a catalyst to evoke collective conversations in the professional culture. Focusing on the issue of accountability, the evocative question for the conversations became: "Might local educators reframe the way they are held accountable by the state testing policy and develop new indicators which more appropriately assess what and how they teach?"

The findings presented in Story 3 include an exploration of why these new conversations did not occur. 162

Story 1: How Did the State Testing Policy Relate to the Conversations in the Professional Culture?

In this section the response of Foster educators to the state testing policy is reported in terms of their conversations as they initially dealt with the policy and prepared to implement it. (As described in Chapter I, the study began ten months prior to the effective date for implementing the achievement and ability portion of the

1987 testing legislation and 20 months prior to the first administration of the state proficiency test.) The findings included within Story 1 address the following specific research questions:

a. How do educators make sense of the policy, including their initial reaction to it?

b. How do local educators see the policy influencing their work and relationships?

c. What rules for discourse shape the state testing policy?

d. What rules for discourse shape the professional culture of local educators?

Prior to discussing these specific findings, a description of what is meant by the Foster professional culture and a characterization of that culture is offered.

In addition, the ongoing communication processes of the

Foster school district and the conversational settings in which talk about the state testing policy occurred are described to provide background for the findings. 163 Professional Culture

The inquiry focuBed on the conversations within the

"professional culture", by which I mean the subculture of the Foster school district which specifically pertains to the interactions among teachers and administrators and which is characterized by a sense of professionalism. Using the definition offered by Darling-Hatnmond and Berry (1988), professionalism refers to "members sharing a common body of knowledge and using shared standards of practice in exercising that knowledge on behalf of clients" (p. v).

Conditions prevalent within a profession include specialized knowledge, self-regulation, special attention to the unique needs of clients, autonomous performance, and responsibility for client welfare. Applied to education, this means professional educators possess and use a specialized body of knowledge as they make decisions about the instruction of students. These decisions may be made individually or collectively and include what should be taught, when, and how, and take into account the developmental and individual needs of students as they grow and mature from kindergarten through grade 12.

By referring to the "culture" of professional educators

I am specifying that subset of the Foster school district organization which is made up of administrators and teachers and the cultural norms which help to shape their actions. As noted previously, cultural norms provide not 164 only for continuity and control within an organization but also provide for the development of identity, integration, and ideals among the members. In Meryl Reis Louis*s words, the culture expresses: "what we as members stand for and how we deal with one another and with outsiders" (1983, p.

44 > .

Within this study I was concerned with what these educators stood for, how they dealt with one another and outsiders, and how the state testing policy was talked of.

By listening to the conversations I sought to uncover how the policy might influence their professional culture and how it would be incorporated into their individual and collective beliefs and practices.

The professional culture was manifested in how members create the "context" for their daily work. The term

"context" highlights the fact that the culture is constantly created and maintained by the actions of the members; it does not exist once and for all, but is continually reinforced by the interactions and relations established between members. The ongoing, unfolding, flexible aspects of the culture are referred to by the term context. In this discussion I move back and forth between the terras culture and context, at one time emphasizing the underlying action-shaping properties of the culture and at another time recognizing that these properties can be changed by altering the context through new actions or new

I 165 conversations by the members.

The characterization of the professional culture within

Foster is based on the interpretations I made from all the different data collection occasions and activities between

January 1989 and March 1990. As summarized in Table 1 and described in Chapter III, these occasions included observing the educators' interactions during 26 meetings and 11 other occasions; interviewing 26 administrators and teachers; reviewing over 100 district documents; and conducting 13 reflection meetings with the key informant.

During these sessions I was attentive to how educators talked to one another, made references to each other in my 1 conversations with them, and seemed to relate to each other verbally as well as nonverbally. Throughout the study, 1 was formulating working hypotheses regarding their relationships with one another.

In general, I concluded that there appeared to be a collective commitment to a culture of professionalism among the educators in Foster. During the interviews and at meetings, administrators and teachers talked freely of the expertise of their colleagues, of the respect they had for one another, and of the support that is provided to try out new ideas and to actively practice instructional approaches suggested by current research. From their accounts of their personal experiences, it seemed that, in general, teachers and administrators were engaged in an ongoing dialogue 166 about their work and itB improvement. Although I would hear of an occasional interpersonal conflict or difference of opinion, it seemed that the overall context for teachers and administrators, at least within each of the Foster schools, was one of respect for their professional expertise and support and challenge for ongoing improvement.

When the focus shifted from the context within schools to the context of the district as a whole, however, there seemed to be less cooperation and more conflict. In marked contrast to the spirit of respect and cooperation evidenced within the schools and across schools in teacher - administrator meetings, the school board meetings included open strife among members and between board members and some community members. Decisions over school attendance boundaries, which students would walk rather than be provided bus transportation, cuts in staff positions, and which field trips were allowed in face of budget shortfalls were made in the midst of controversy, accusations, and, for some board members, threats.

In between the context of a professional culture created within schools and that of open strife created among board and community members sit the central office and building administrators. The context for their work seemed to be marked by tension, characterized by a feeling of isolation from other administrators. For example, when 167 talking to me about the state testing policy, a number of administrators expressed the opinion that their views probably seemed "radical" or unusual from those of the other people I had talked to. In fact, I was surprised by the remarkable similarity among the views I was hearing. I then began to wonder why principals and central office administrators were not aware that their counterparts shared their perspective about the state testing policy.

The administrator meetings 1 observed seemed to be more strained and less free flowing than those among teachers or between teachers and administrators. For example, at a meeting to discuss the test results in May 1989, there was a beginning of a conversation regarding why it was that the instructional objectives measured by the California

Achievement Test matched so few of the instructional objectives in the Foster course of study for language arts.

Although this question was raised by one principal, the other principals and central office administrators did not respond. As an observer, I had the sense that this was an

"old conversation"; that is, one in which people had engaged in before and did not have the interest or energy to engage in again. The lack of response was filled with a strained silence, almost as if moBt people were hoping the topic would drop rather than cause controversy or consternation for which no one had the energy. The overall tone seemed to be characterized by fatigue, tension, and an 168 urgency to get the meeting over with as soon as possible.

In general, it seemed that principals appeared to have developed individual havens in which teachers could enact their professional roles within each building, yet they did not seem to have a similar professional haven across buildings for themselves. Understandably principals are in a position to feel the press of criticism and strife from the community and school board. Yet some principals appear to feel alone in this tension, and not a part of a larger context of support from their colleagues across the district.

Recognizing that these were high inference interpretations on my part, I directly asked three principals and two central office to read this characterization of the Foster professional context and verify its accuracy. All five concurred and I then included this description in the introduction to the 45-page summary of the findings given to all participants in February 1990.

Subsequently, another central office administrator told me that my description of the contrast between what occurred in individual schools and what occurred across schools and with the central office was accurate and added that the tendency for principals to focus only on their buildings and not on the district as a whole was getting stronger as the strife in the community and with the school board increased. 169

An additional aspect of the professional culture is the

role chosen by the superintendent. In general, his primary

focus seemed to be on the facilities, finances, and public contact aspects of the leadership role. Curriculum and

instructional activities appeared to be left mostly to the assistant superintendent, the central office staff, the building principals, and teachers. Given the rapid growth

of the district, and the need to attend to the building and

financing of new schools and to the concerns of a constantly changing public, this is an understandable

choice on the part of the superintendent. Such an emphasis,

however, has implications for the school district

conversations about a new state policy.

The 1987 state testing policy entered these various

contexts of the Foster organization through the

interpretations given it by local actors. Whether and how

this policy changed the ongoing actions and interactions

depended on how administrators and teachers made sense of

it and acted on it. The assistant superintendent for

curriculum and instruction had the responsibility to lead

the district’s response to the state policy. The meaning he ascribed to the policy was therefore central to this study.

He was most responsible for communicating information about

the policy to others in the organization. He was also

responsible for framing and organizing the actions of

teachers, building administrators, and central office 170 administrators to fulfill the state requirements. He thus became an important conversant in the talk surrounding the state testing policy.

Communication Processes and Conversational Settings

A description of the methods of communication used within the Foster school district is offered as background to the specific conversations associated with the state testing policy. Not unlike other organizations, the Foster school district communicates by holding meetings, recording minutes, sending memos, and making presentations. It appoints committees to study and recommend actions for long range and/or ambiguous issues. Informal conversations take place before and after formal sessions, and individuals have differential access to others based both on their formal roles within the organizational structure and on their informal relationships.

The emergence of the state testing policy evoked each of these patterns of communication. A dormant district

"testing committee" was reactivated during fall 1988 to discuss and decide upon Foster’s response to the achievement/ability testing mandate. After comparing each of the tests on the state approved list with the district curriculum, the test which they had been using, the

California Achievement Test, seemed to be the closest fit.

During the fall the decision was made to purchase the 171 computer hardware and software to score and analyze these tests locally rather than send them off to the commercial publisher for scoring. This decision engendered further conversations and visits to other districts to find the best hardware and software for their needs. Ongoing conversations during the winter and spring were needed to negotiate with vendors in order to get the scoring and analysis system up and running in their district with their data.

In early April 1989, the assistant superintendent presented teachers with instructions for administering the achievement and ability tests through relatively formal meetings, described as inservice sessions. He encouraged them to talk further with their grade level counterparts, principals, and special education and English as a Second

Language teachers to coordinate the testing schedule and to make provisions for students needing special circumstances for the late April testing. At the same time, he held meetings with central office administrators to determine the formats of the test reports for parents, principals, teachers and the school board. In May, the assistant superintendent gave the test results to principals at a meeting held specifically for this purpose. The topics for discussion included how the principals would forward the scores to teachers and what use would be made of the results. Careful preparations were made for the formal 172 presentation of the test results at the mid-June school board meeting. Both the written and verbal aspects of the presentation were designed to convey information about student performance on the test without inviting what administrators believed to be unfair comparisons among schools.

In contrast to the "information-passing" character of the communications regarding the achievement/ability tests, the processes used to communicate about the proficiency tests were more open ended. Eighth grade and high school teachers of the four areas tested (English, reading, mathematics and social studies) were invited to an early

February 1989 after-school meeting to discuss the recently promulgated state wide "learning outcomes" which would be assessed on the test. Given that the meeting occurred in the same week that 106 teachers received layoff notices (to be activated if the emergency operating levy did not pass), the assistant superintendent did not expect more than ten teachers to come. Over 20 teachers attended and participated in a lively discussion about the test and its potential impact on their teaching and their students.

Individual teachers remained after the meeting to talk informally with the assistant superintendent about the testing and other curricular and instructional issues.

Ab a result of this meeting, the assistant superintendent sent a series of memos asking the affected 173 high school departments and the middle school eighth grade teams to hold meetings in their buildings during which the teachers would complete a form for each of their courses noting if the state's learning outcomes were taught in the course. Once they knew how well their current teaching practices matched the state's learning outcomes, it was felt that they would be in a better position to decide how best to further prepare their students for the state test.

The decision regarding whether to make adjustments to their written courses of study or their instructional practices because of the upcoming state proficiency test remained a point of discussion among the teachers and administrators in the district more than a year after this initial meeting. The decision was part of the year-long deliberations of the language arts/reading course of Btudy revision committee; it was repeatedly raised in a special university course designed for Foster teachers to revise their approach to teaching mathematics in the middle and high school grades; and it was the reason for calling special sessions of social studies teachers in the middle and high schools in January and February 1990.

These meetings, inservice sessions, and presentations made up Borne of the naturally occurring conversational settings during which I could observe the substance and form of the professional talk about the state testing policy. There were additional occasions, however, such as 174 conversations when two people would talk in formal or informal settings, and meetings where the state testing policy was only one topic on a larger agenda, when I was not able to observe the conversations. As noted earlier, in addition to attending 26 meetings among educators and the local school board, I initiated interviews with 26 different administrators and teachers to ask about the effect of the state testing policy on their work and their relationships.

The remaining portions of Story 1 recount the Foster educators’ response to the state testing policy in terms of the specific research questions.

Educators* Sense-Making and Reactions to the Policy

a. How do educators make sense of the policy, including their initial reaction to it?

As described earlier, language is the predominant symbol in any culture and organizational members constantly create and maintain their meaning systems through their communication. As Thayer maintained: "We say the structures and arrangements of our world into existence" (1988, p. xi, emphasis in original). Because members are constantly confronted with new information from the environment which challenges their perceptions of the organization, they are continually engaged in developing consensual meaning to 175 keep creating and maintaining their shared sense of organizational reality. As expressed by Mumby (1988):

A taken-for-granted social reality is created in an organization through the continuous movement back and forth between sense and non-sense. In other words, social actors must frame ambiguous information in terms of what they already know "makes sense." This ambiguous information, in turn, subtly changes an actor’s perception and definition of "organization reality", (p. 10)

The state testing policy is an example of ambiguous information confronting the organizational reality of the professional culture within Foster. How the members of this organization frame this ambiguous information and how it, in turn, subtly changes their perceptions of what constitutes their organizational reality is described in this section of the Btudy’s findings. By focusing on the conversations which surround the initial sense-making related to the state testing policy we can see how this external mandate is enacted by the members of the professional culture within Foster.

Two overall findings pertain to how Foster educators made sense of and initially reacted to the 1987 state testing policy: 1) they were ambivalent about its potential to be beneficial and/or harmful to education; and 2) it created a professional dilemma for them. 176

Ambivalence

For most Foster administrators, the pervasive reaction to the state testing policy was one of ambivalence. On the one hand, they could see some benefits to testing. On the other hand, there were serious concerns about its adverse consequences for schools in general, for their instructional program in particular, and for what it meant for them as professional educators and for their relationships with each other and with teachers. The following quotes from various central office and building administrators portray this ambivalence;

Building administrator: I'm not a real strong proponent of standardized testing. ...But it has it's -- in it’s way, it has Borne benefits and so I think at this time it’s probably our best alternative until we can design a test that tests our students on what we have taught. ...I'm not a real strong proponent of [testing], but there's a place for it. And, I don't mind taking the time out of the year,...if you look at the whole scope of things, 8 hours at the max — 8 hours out of the school year -- it's well worth the time.

...But there’s an innate comparison between Foster and [neighboring districts] and , . we try to minimize that the best we can. Not that we're ashamed or afraid that we'd come up short, I just ...don't think it's good for education. ..Basically, I know we'd stack up well -- I just hate the thought of it becoming a war -- our school district did this and yours did that.

...I guess my whole theory of testing - there's a place for it, and I see a lot of reasons for it - but I juBt hope that it’s not the Bole basis of looking at a school or a kid and saying, well, that's just the way it is, or that's a great school because they teBt real well, or that’s a bad school because the kids didn't. I guess my overriding fear of all this testing is the ...comparisons; using it to be the sole judge or evaluation of a high school program or a middle school program...I have enough faith in our people who are 177

making that decision that I think that’s not going to happen, but...I just hate to see the test being a ranking-type figure.

Building administrator: I understand the [state’s] motivations. I think they’re pure motivations. I don’t think they’re bad motivations. I think people are saying, ‘Let’s see if our kids cut it across the state and can compete’ and 'We want assurance that the school systems are addressing the issues’...I ’m not sure that they can design a test to do that.

...Maybe we could design a way that you could truly validate what we’re doing in Foster or what someone else is doing in [neighboring district], or someone else is doing....but it’s not going to be across the state, because again you’d be dealing with what does the community value, what does the school system value, what do the teachers, students, and families value and what is actually being taught.

...And who’s to say what absolutely needs to be taught at that grade level. Our graded course of study would probably differ from [other districts] at certain grade levels. Which one of us is right? Probably all of us.

Assistant superintendent: There are too many confounding variables to attribute school success to an achievement/ability test. It’s really an inadequate profile of learning and in Borne cases it is actually at odds with what you're trying to teach.

...So, consequently, I’ve sat at board committee [meetings] and said time and time again that, remember folks, we ’re talking about three hours [of testing] out of hundreds and hundreds of hours [of schooling].

...We’re talking about a testing program that is developed to discriminate among students and it’s something you can be very cautious about, that is, taking an aggregate score and judging a building, or judging a class or certainly judging a teacher.

,..And we also believe that you cannot afford to do poorly on these norm-referenced tests, I mean, are these curricula so idiosyncratic that it excuses you 178

from doing well? No, I don’t think that Foster is that unique a community. I don't believe that, but by the same token we want to foster in the instructional leaders the attitude that this is just a snapshot in a big portfolio of student work.

...And if there's something that’s really out of sync in this third grade teacher’s room, and you didn’t expect it..then you have a right to pursue it. It’s not a big stick, it’s just a red flag. ...Not any differently than what we say to a classroom teacher. "We expect that this will confirm what you already know. If it doesn't, it’s something you want to probe".

...I am very uncomfortable serving this role to the district, I don’t enjoy it one bit. It forces the faculty, including administrators, to see me as an agent of the state.

...It does affect, and I think in many cases, pollute your relationship with faculty. And I don’t enjoy it at all.

Central office administrator: I was a part of the committee that went around looking at what other districts are doing and I just walked away thinking, - my big question is ’Why’?. Why all of that? What is it doing for the kids?

...I see that it really does nothing for the kids but make them anxious and make the parents anxious. And many times it doesn't even give us complete information, better information than we have on them from teachers daily notations in their grade books and notations on observation cards and things like that.

...I think in this whole policy that kids were left out of it. In other words, we have to suffer with it.

A part of local educators’ ambivalence was reflected in their expressions of long term concern over the effects of the policy coupled with some short term benefits for the district. The short term impact of the state regulation was that it provided the impetus for Foster to reorganize and 179

exercise some control over their district-wide testing program. By using available capital funds to purchase

computer hardware and software, they could develop the

internal capacity to score, analyze and report the

achievement and ability tests required by the state as well

as set up the potential for scoring and analyzing criterion

referenced tests to match their local written curriculum in the future. The ability to control the testing tasks within the district was seen as an advantage to being dependent upon the timing, analyses, and reporting formats offered by the test publisher, all of which had proved inadequate for their needs in the past. This ambivalence was expressed in an exchange between myself and a central office administrator as follows:

Central office administrator: Well, probably the one disclaimer I ought to toss in is that, while the state regulations have driven this a lot -- and that’B not all bad in that respect -- we have needed to do something in terms of district-wide testing for a long time...I don’t know without the state regulations where we’d be today and what kind of direction w e ’d be...Obviously it’s gained some leverage for us, in terms of getting the [hardware and software] equipment we need right now.

So there’s some positives there... I think we're concerned about the fact there’s so much focus on the standardized testing and proficiency tests coming up and those kinds of things, as opposed to the emphasis — what was it, five years ago — on the Competency Based Education testing. I think we ’d better embrace that — we prefer to embrace that than this. But this has had a lot more bang for making change than the other regulations do. So there’s some good there, too, I think. The key for us is going to be what happens down the road. 180

Researcher: Well, one of the things that I was immediately struck by when I first started talking about this is the sense of making it your own. I mean, yes, this is state regulation, but obviously a lot has happened in Foster that has taken the state regulation and said, okay, how can we make it work for us?

Central office administrator: How we can make it useful.

In addition to taking control over the testing process, there was the view that the norm-referenced achievement test offers some useful information regarding how the local curriculum compares to that of other schools across the country. As discussed at a meeting among administrators:

Principal: Only one of 15 language expression objectives [on the test] correlates with our graded course of study...This kinda catches my eye...There are good indicators here. What all this does for us is tell us how close to our anticipations we are actually performing,..

Researcher: If the test doesn't match the objectives in the language arts course of study, do you reconsider the course of study or the test?

Principal: It serves as a red flag for something to consider; you don't go changing either one, but you look.

But these tests are the way they are because three-fourths of the country thinks this is what fourth graders ought to know; we should know how our kids do on that. We aren't that different.

I believe that if kids are taught with one approach they should be able to do the skill even if it is presented to them differently- if they really know it they will be able to do it.

Another principal: I was surprised at your answer to her question [paraphrased as "do you change the test or the course of study"]. I expected the automatic response to be, you change the test. 181

Although this difference of opinion regarding the value of the test was important to some principals, for another the state testing policy was not the most important aspect of education to focus upon:

Principal: Well I think I Mould respond to the [state testing policy] by giving you my overall philosophy of life, and that applies to policy as well as every other avenue of my life. I think you have to separate out that which is under your control and that which is not. And if it is within your control, then you need to work hard to affect change, to do whatever needs to be done to make the best situation. If it is not under your direct control, then I think you need to do the best that you can to work with it; to look at the advantages of it, to look at the positive parts of it, and then you work it in to your own scheme and make it a positive thing.

The state testing program is in many ways out of my direct control. Now if I felt strongly that it was not headed in the right direction, and that something really needed to be done about it and that I personally needed to be involved in it, there’s some things that I could do.

That is not one of my priorities. I can see some advantages to it and I can see some of the disadvantages, but it’s not one of the hills that I choose to stand and fight on, and put my energies to. So I'm catergorizing this as basically something that I ’m not choosing to take an active part in changing. Therefore I ’m going to work with it and do the best that I can do and so having a positive view of what can come of the policy, it doesn’t adversely affect my relationship with anyone because of my personal view of, ’Well, this is what we have now, therefore let’s look at the advantages and let’s work with the program and try to make the best sense we can of it,’

This feeling of being stuck with the policy so let’s try to make the best of it was echoed by a number of other

Foster administrators: 182

Central office administrator: The compelling reason is that we have got to do the testing mandated by state law. Given we have this directive, what sense can we make of it; what help can it give us?...The immediate payoff is that some items [on the test] do fit the course of study and this will be useful information for teachers if we can get it back to teachers in a timely manner.

...Also, we need to have accountability. We have to be able to give people information when we are being criticized for schools costing so much money and having such beautiful facilities.

The testing committee’s attitude was, ’Well, just because we have to give the standardized test, we ought to try and glean as much information as we can for instructional purposes ....[The assistant superintendent] sat down with a lot of u b and said ’We've got to at least head this thing in the right direction.’

Principal: I look on testing as just one more thing I ’ve got to do; not that I want to do, but have got to do.

As noted earlier, the Assistant Superintendent for

curriculum and instruction was responsible for implementing

the state testing policy in Foster. His own sense making of

the achievement testing is reflected in the following

statements made during inBervice sessions preparing

teachers for the administration of the tests:

Assistant superintendent: I'll anticipate your feelings. We know you have a positive atmosphere in this building - we want this to continue. We didn't have anything to do with requiring this test, but it is required and we want students to do their best This is required, there is some useful information though..we'11 get [test information] back to you quickly this year. The items correlated to our course of study will be asterisked... 183

...We know that this [test] doesn't do what you want it to. It is mandated; we have no choice; it's legislated learning in its classic form, ,.We have gone through the test and the course of study item by item to see which items match...

...Retain as much of the classroom environment as you can. Don't distort the classroom atmosphere. This test does enough of that already. You are the best ones to determine [how to do] that. I wouldn't presume to tell you...

...We would not give this [sub]test in Foster if we did not have to...The language mechanics test is all capitalization and punctuation. The language expression (I love that term) is all usage. We are required to do this by the state because we have to submit a discrepancy score between ability (from the Test of Cognitive Skills) and achievement (from the California Achievement Test) in language, reading and mathematics. So we have to give this [language] test in Foster although we never have done it before...

...It's not the complete battery [of CAT tests], and it's much less than in our neighboring districts...

...This data’s getting reported to the state, and it's going to be in the papers. ...Until the legislature changes its mind about this testing, this is going to be a big deal.

By "big deal", the assistant superintendent was referring to negative consequences from the public if students do not perform well on the tests. This sense of impending consequences created a dilemma for Foster educators, as described more fully below. In the following example of his sense-making, the assistant superintendent is being confronted with the question of whether the district will change its written curriculum and its instructional practices to align them with the upcoming state proficiency test. At a February 1989 meeting in which 184 high school teachers are considering the state defined learning objectives for the proficiency test, the assistant superintendent and a teacher had the following conversation:

Teacher: So we will be teaching to the test?

Assistant Superintendent: We will be teaching to the test. This starts to demand a common core curriculum since we can't deny any student the opportunity to pass thiB test and get the highest diploma.

...What about kids who take 22 credits of courses and don't pass [the proficiency test] in a community of lawyers - it's [politically] indefensible. This is a nightmare.

...This is a classic case of curriculum being dictated by virtue of the test driving it.

Teacher: It's scary.

Assistant superintendent: High school graduation depends on this, so we can’t afford not to teach it.

Another example of the sense-making of the assistant superintendent was a series of memos he wrote in early 1989 regarding Foster's response to the state proficiency testing. Upon reviewing them, I perceived a change in his stance over the course of the three memos. They seemed to

Btart with a questioning of: “How shall we handle this?" and move quickly to a more definitive: "We must teach to the test":

1-11-89: Memo inviting eighth grade and high school teachers to a meeting: "to discuss guidelines, strategies, etc. that we should implement to ensure our students’ preparedness for these tests." 185

2-2-89: Minutes from the meeting noted above: "The group as a whole decided to distribute the list of [learning outcomes for the state proficiency test] to every 8th and 9th grade teacher in the district to determine which outcomes are taught and in what subject and grade level."

2-10-89: Cover letter accompanying form listing the state learning outcomes and asking teachers to indicate whether or not they teach each outcome within a particular course: "The next step is to convene a group of interested teachers to develop strategies to ensure that students are taught each learning outcome in preparation for the 9th grade proficiency test. Although this process dictates 'teaching to the test,' there is no real alternative if we want to enhance our students' chance for success." (emphasis added)

When I asked about his apparent movement, from "how might we deal with this?" to "we have no alternative but to

teach to the test", the assistant superintendent responded:

Assistant Superintendent: There are levels in this. At one level, there's the political reality of all high school students taking proficiency tests in the state of Ohio, and the absolute untenable position for Foster schools to have its students come out in profile as doing poorly. It’s just not politically acceptable. And I'm working with the superintendent and the board [who] made it clear to me that •We’ll find a way, won’t we, to do well on these tests.' ...Not in any heavy-handed way. It’s, rather, you know — •The challenge is at your doorstep, now make sure that we do well. Okay?1 And, frankly, I don’t believe that any district in Ohio ...is assuming a much different posture, in the sense that people realize there's going to be testing, and it would be in their best interests for students to do well. That's one level.

[The next level] is 'okay, what can we do?' ...That level involves a lot of choice, and I’d like the educators, the teacherB, in our classrooms to make those decisions. I don’t want to do that, because they know the best way how to accomplish those ends. So there is a level of choice. 186

...The option of whether or not to take the test is not there. It's not an option. To my way of thinking, the option of whether to do well or not to do well, or whether to run the risk of not doing well -- why address it? It*B not an option.

...It's a local and perhaps a regional and national issue right now, and that's the matter of accountability. I mean, for the state to administer what they regard as proficiency...test, and for Foster Btudents not to do well?

...I want to minimize this as much as I can — put it out there, we're required to do it, we ’re going to collect some information to see where we're at. "

This conversation illustrates what Anderson (1990) referred to as one of the mediating task of school administrators. Not only do administrators have to mediate open conflict among different parties in the district, and mediate on a day-to-day basis how meaning is managed in the district, they have to mediate conflicts within themselves.

Using Abravanel (1983) distinction, this inner struggle usually revolves around the contradiction between the administrator's ideals (allegiance to fundamental moral principles) and operative concerns (practical and immediate considerations for survival). In order to be able to act, the administrator must resolve this inner conflict. As

Anderson put it, the administrator must engage in the

"cognitive task of resolving (or perhaps dissolving) contradictions within the structure of one*8 own ideology"

(1990, p. 47). 187

In this instancet the assistant superintendent has to

resolve his own professional discomfort with teaching to a

state test. One way to do this is to create different

"levels" in his understanding of the conflict. These levels allowed him to distinguish between the top-down state mandate (which left him no choice but to respond) and the

local culture of teacher professionalism (which allowed him to let teachers choose how to respond). Having constructed

these levels within his mind, he was able to act in a way which simultaneously fulfilled both his legal responsibility to carry out the state mandate and his personal philosophical responsibility to maintain a professional culture for teachers.

Thus the sense making for Foster administrators

involves handling their ambivalence over the potential good and potential harm they feel may come from the state mandate to administer the achievement and proficiency

tests, and their own inner dilemma over responding to a mandate that they do not feel is necessarily in the best

interests of their students or education in general. The next section describes this professional dilemma in more detail. 188

Profesaional Dilemma

Throughout the conversations of Foster educators, there was one refrain which captures their reaction to the state testing policy. It can be summarized as follows:

State testing could make educators lose sight of their central purpose: doing what is best for students. If educators have to base their daily decisions on what makes them look good to those above them in the schooling hierarchy, instead of on what their professional judgement tells them is best for students, everyone loses. Even the very best teachers and administrators cannot resist forever the pressure that comes from comparing test scores.

This was the most pervasive finding in the research. It represents a "core category" in Strauss's term, that is, it was the "main theme...the main concern of or problem for the people in the setting,...it sums up in a pattern of behavior the substance of what is going on in the data"

(Strauss, 1987, p. 35). The unsettling dilemma of whether the state testing will evoke actions in educators that they feel are not in the best interest of their students is manifest in their discussions of all the issues surrounding the testing policy.

Administrators have a particularly hard time with this dilemma because they are the ones obliged to enforce the policy, a policy with which they disagree, at least in part. How do administrators justify this testing to teachers when they cannot justify it to themselves? As put 189 by the assistant superintendent when introducing the achievement testing to teachers: “I'm not justifying this,

I'm just describing [it]."

One aspect of this professional dilemma pertained to what the state test might do to the local curriculum, a curriculum that was developed by local teachers to respond to the needs of their students and which reflects their professional judgement regarding what should be taught and how. The following quote illustrates the personal struggle of one building administrator as he tried to explain the policy and considered its effect on both the local curriculum and the professional relations within the school:

Building administrator: "I don't think Foster, Ohio needs to be accountable to the State Department of Education as far as what their kids -- what knowledge base they have. We need to be accountable to the community and the people that live in Foster, Ohio, basically."

...They [the teachers] are really not negative about it [the proficiency test]. You know, they're just saying that, 'I'd rather be doing what I'm supposed to be doing.'...Because they know that [the principals] are looking at them and making sure that they are covering the graded course of study, and now we have to do this other thing.

...And I think it's hard for them to justify it, to be quite frank with you, I really do. I really think that they're struggling with that. It's hard for me to justify doing this. I'll do it because you tell me to do it, but professionally it's hard for me to justify it.... I think, too, if we could show them [teachers] what benefit this is going to have to ray class and my students -- if we could show them that, if we had something to show them, then ...they would have no problem with it. I guess as an administrator I’m kind of struggling with that fact. What benefit is this all 190

going to give us? I'm not so sure I know. But I think the whole idea behind this, from the Btate department...was accountability, which, again, we need to have. But I'm not so sure the benefits are going to be what we want, or even what we need.

...And the change process is a slow process, and you just can’t force things upon people. You have to -- it's a part of leadership. They have to accept it eventually, and, you know, and want to do it. And I think, again, this goes back to the state testing, too, you can relate it to that. You can’t -- you can’t force things on people. You can't force things on professional, educated people. You just can’t do it. It has to evolve over time, and they have to accept the change and want to do it. I don't think w e ’re there... I really don’t."

The personal struggle of the assistant superintendent regarding the proficiency test was manifested in the following conversation at the same high school teachers’ meeting noted above:

Teacher: We'll need to think about remedial work for those who don’t pass it.

Assistant superintendent: I don’t mean to be flippant about this, but it is a nightmare.

Teacher: Are you for this or against this?

Assistant superintendent: I’m staunchly against it and testified to that effect because I ’m in favor of local control. The state just went through revising minimum standards and telling us to develop competency test to fit our own locally developed courses of study. And this is going to force us to change our curriculum to fit these outcomes (which are state determined).

The teacher’s question ("Are you for this or against it?") highlights the policy's affect on the assistant superintendent's professional relationships with teachers. 191

As he later told me during one of our reflection meetings,

ordinarily he can serve as a professional colleague to

teachers; as one who is there to assist them do what they

know in their professional judgement is best for students.

This conversation revealed how the mandated state policy makes him act, in this instance, as an "agent of the

state", his words to describe this role. He disliked this

role and saw it as interfering with his professional

relationship with teachers.

Another aspect of the assistant superintendent's dilemma was revealed in the following conversation which

focused on my written summary of the issues raised in the high school teachers* meeting about the proficiency test:

Assistant superintendent: ...I think [the summary] is an accurate reflection of the meeting, and I think they're relevant questions.

Researcher: Okay. Is my doing that kind of summary useful to you?

Assistant superintendent: It’s also exacerbating to me, as well.

Researcher: Because it brings up issues that have no resolution?

Assistant superintendent: Yes, and frankly ...I choose to submerge some of that stuff because it's incapacitating to me .

Researcher: And you've got a way to...

Assistant superintendent:.,, it's debilitating to me. You can't...I mean, those are rich educational questions, and I don't think that this testing is an educational question. It's a political question. And I ’m not naive enough to separate the two. I mean, obviously they are inextricably connected, but what is overriding 192

here, what is compelling, is the political issue of school accountability in the economic machine.

Researcher: Then if I raise the educational issues, and the teachers raise the educational issues, or you deal with the educational issues with teachers, it makes it more difficult to simultaneously deal with the political issues?

Assistant superintendent: It’s tough to attend to the political reality — that’s right.

Researcher: So you either can attend to one or the other, but if they're incompatible. . .

Assistant superintendent: That’s right, and. . .

Researcher: ...so you’re the one who gets squeezed in the middle.

Assistant Superintendent: And they [teachers] are dealing with it in a de-politicized context, in the sense that they’ve had relatively little input into the whole process — even though we made them aware, and so on -- as a profession, even though OEA [teachers’ association] might have been active, these teachers, by and large, with rare exception, were not [involved] -- it’s a done deal. It’s fait accompli. They’ve got to do it. You know, it's top-down. It's all that we read about in the literature, ...So, to engage them in that kind of discussion, I just think it's counterproductive, I really do.

Researcher: And at times it’s counterproductive to engage you in it, too, because it puts you in a situation where you can’t move — and you have to move.

Assistant superintendent: Right. You’re right about that.

Researcher: Okay, so this reflection may be more painful than helpful.

This conversation provides another illustration of what

Anderson (1990) referred to as the cognitive task of mediation for the Bchool administrator. The administrator has to mediate a conflict within himself before he can 193 mediate a conflict in the organization. In this case the conflict is between the educational questions raised by the learning outcomes of the proficiency test and the political reality that their students cannot do poorly on this test.

Even if teachers and administrators do not agree professionally with the educational content of the test, politically they must insure that their students score well on it. So the assistant superintendent, despite his responsibility for curriculum and instruction in the district, must "submerge" the educational questions beneath the political questions in order to be able to act, to do what he has to do, in his capacity as an administrator in the district. Again, in Anderson’s words, he has to

"...resolve (or perhaps dissolve) contradictions within the structure of one's own ideology” (p. 47).

The final aspect of this dilemma pertains to how teaching will change in order for educators to appear successful to those outside of the school who are judging them on the basis of the test results. As revealed in this conversation with a school principal:

Principal: If we start to drive the curriculum in order to do well on the test, then I see us doing more of the things that we've moved away from...the workbook kind of things, the isolated skills, the skills...we don't feel are grade- or age-appropriate.

...I don't want us to start to allow the tail to wag the dog, in that the test dictates what our teachers are doing. And we can tell people till we're blue in the face that that [test] is meaningless, that we're 194

doing something else, but even our own teachers if they see their scores come back [low]...

Researcher; The notion that the test could drive the curriculum, even among the best teachers..?

Principal: Absolutely... it'b going to, it's not a matter of *if'; it will. If I were a very good classroom teacher and I knew that at the end of second grade my students were going to get a test on grammar and that that test included semicolons, even if I felt semicolons were inappropriate second grade material, I ’d cover it.

Because I wouldn't want the students to look bad or me to look bad. And I think that’s what’s going to happen.

The concern over what counts aB success is exacerbated with the state's intent to compare schools and school districts on the basis of these test scores. As put by another administrator:

Principal: If these statewide or regional comparisons are made, we are going to start teaching to the test, which is going to take us off our home base of teaching what is best for kids."

In their articulation of the conflict between what the state testing policy seems to be asking of them and what they feel they should be doing as education professionals, the Foster administrators and teacherB reveal that the policy is challenging what they consider "sacred norms". As defined by Corbett, Firestone, and Rossman (1987), sacred cultural norms within schools are those associated with professional purposes; they are what define their identity; they provide the personal reasons for being educators: 195

Attacks on the sacred undermine professional identity and call the meaning of teaching into question. Proposed changes, then, challenge more than "the way we do things around here" [profane norms]; they also threaten "who we are around here" [sacred norms] (p. 56).

In summary, Poster educators’ sense-making and initial

reaction to the state policy was one of ambivalence between

acknowledging the need for accountability while disagreeing

with this form of accountability. It created for them a

professional dilemma in which they felt they had to choose

between doing what their professional judgement said was

best for students and what would make them "look good" in

the schooling hierarchy and to the public.

Our attention now turns to why the state testing policy

was seen as so important to Foster educators. The second

research question focused on the effect of the state

testing policy on the professional work and professional relationships in Foster.

Impact on Professional Work and Relationships

b. How do local educators see the policy influencing their work and relationships?

As described in Chapter III, the analysis of interview and observation data resulted in my organizing the perspectives of the Foster educators around six topics or 196

"themes". In general, the concern regarding the anticipated

impact of the state testing policy focused around the

following six themes:

1) the use of the test results;

2) the effect of testing on relationships;

3) the benefits of fall vs. spring achievement testing;

4) the effect of testing on the local curriculum;

5) the effect of testing on local resources; and

6) the issue of local and state accountability.

The first five themes are described in detail below and are summarized in Table 2. The sixth theme (accountability) is attended to later on in this chapter as a part of Story

3. These themes were the subject of two comprehensive member checks during the course of the study. 197

Table 2. Themes 1-5: Anticipated Impact of State Testing Policy on Profeasional Work and Relationships

Theme 1: UBe of TeBt Results

Potential Benefits of Achievement Test:

- internal control over scoring, analysis, and report formats; - alignment of test with some aspects of reading course of study; - useful results for some specific programs (special education, gifted and talented); - specific benefits for individual students; - provides one form of accountability information to local community.

Potential Misuses of Achievement Test:

- if used to compare teachers, schools, and districts ensuing competition could harm rather than enhance education of students; - if viewed as valid and relevant measure of local curriculum and instruction when in fact there is a mismatch in most subject areas; - if tried to base instructional decisions on test results when test is designed more for political purpose of accountability than instructional purpose of assessment.

Theme 2: Effect of Testing on Relationships

Relationships Among Educators:

- teachers may see administrators as giving them contradictory directions regarding what to emphasize in their teaching; - administrators may be tempted to evaluate and compare teachers based on student test results; - teachers could blame each other for poor student performance on tests; - teachers may not want to teach interdisciplinary units if their teaching will be Judged according to student scores on isolated subject areas; - if teBt scores are the basis for comparing schools, principals could be placed in competitive rather than supportive relationships with each other. 198

Table 2 (Continued)

Relationships Between Teachers and Students:

- emphasis may shift to "test-oriented" from "student - oriented" instruction, especially in middle schools; - standardized test conditions do not allow students to receive clarifying help from teachers.

Relationships Between Educators and Local Community:

- state required tests may be viewed as appropriate measures of local schooling; - explanations from local educators regarding inappropriate nature of tests may be viewed b b defensive and sb a way of avoiding accountability; - unfair comparisons between teachers and schools could be made on basis of test results.

Theme 3: Benefits of Fall vs. Spring Testing

Fall Testing:

- use information from CAT regarding student mastery of specific objectives for instructional decisions; - students are inclined to take tests more seriously in fall than in spring;

Spring Testing:

- spring test scores would be higher than fall scores when comparisons are made with other districts; - fall testing would catch students before they had overcome learning loss from summer and were back into school routines; - spring testing would be useful for evaluating instructional programs; - fall achievement testing would interfere with more instructionally relevant fall competency testing.

Theme 4: Effects of Testing on Local Curriculum

Proficiency Testing Concerns:

- whether test is beneficial to students is seen as a non-discussble issue for educators; - whether and how they might "teach to the test"; 199

Table 2 (Continued)

- whether a state test or a state curriculum could meet diverse needs of all Ohio Bchool districts, and whether legislators should dictate curricular and instructional decisions; - test could focus teachers on content rather than on students, especially at the middle schools; - curriculum revisions might be necessary for students who do not pass proficiency test; - differentiated diplomas may have serious consequences for life chances of students.

Achievement Testing Concerns:

- lack of match with local curriculum will bring a change in local curriculum; - test could drive instructional practices back to outmoded approaches; - valuable teaching time will be diverted to testing; - timing of testing could affect teaching of interdisciplinary units in middle schools; - disruption of scheduling and teachers* reservations about test could affect student scores.

Concerns Regarding Proficiency and Achievement Testing:

- testing pulls scares resources away from instruction; - teaching toward the proficiency test could hurt student performance on achievement test and vice versa.

Theme 5: Effect of Testing on Local Resources

Achievement Testing:

- over 50 substantive and logistical tasks required of different district personnel to accomplish testing; - over $48,000 in local funds used to purchase hardware,’ software, and required testing materials.

Proficiency Testing:

- ongoing meetings among educators across disciplines and grade levelB as well as within subject areas to deliberate on Foster response to learning outcomes specified for proficiency test. 200

Theme 1: Use of the Test Results

As noted above, the state testing mandate provided the impetus to consider how district wide testing could be locally useful. The Foster response to the mandate was

"given that this is required of u b , how can we make it useful to our district?". With this orientation, the achievement/ability testing was done one year ahead of the state deadline and grade 3 was included in the Foster testing in addition to the required grades 4, 6, and 8.

(Grade 3 was included to provide Bcores for selecting students into the local gifted and talented program.)

Furthermore, the district purchased hardware and software to score and analyze the test locally rather than annually pay for these services from the testing company.

The California Achievement Test (CAT) was selected as the test most closely aligned with the Foster reading curriculum 1 . The associated Test of Cognitive SkillB

(TCS) was chosen to provide the "ability" score required by the state testing policy.

The findings for Theme 1 are presented in terms of anticipated uses of the achievement test prior to its administration in April 1989 and then the actual use made of the test scores when they became available in May 1989.

Both before and after the test administration, Foster educators expressed hopes for how the tests might be beneficial and concerns over their potential misuse. These 201

hopes and concerns are described below.

Potential benefits. At least five benefits were

anticipated from the testing policy. First, by taking

internal control over the scoring, analysis and reporting

of the results, Foster educators hoped to make the test

information available in a timely fashion and in a useable

format. By acquiring the hardware and software to score and

analyze the tests within the district, rather than Bending

them to the publisher’s scoring service, the results could

be available before the end of the school year and could be

organized to provide a variety of information to teachers,

principals, parents, and the school board. (For example,

the scores of sixth graders initially could be organized by middle school teams for the end of the 1988-89 school year

and then could be rerun and reorganized as seventh graders according to the new team membership for the fall of

1989-90. This would make it possible to give seventh grade

teachers a roster of scores for their incoming students.

Such a roster could be broken out by subject matter as well, so that, for example, the math teacher receives just

the math scores, etc.)

Secondly, given the relevance of some of the reading

test items to the Foster course of study, the tests could be useful for teachers and others to see how students perform on a nationally normed test. Because the locally prepared test reports could identify exactly which sets of 202 test items correspond most closely with the Foster reading curriculum, the local reports could provide useful information for examining the teaching of those skills. In addition, with internal scoring and analysis capability, the tests could give some program evaluation information which had not been available before. For example, contrasts could be made between those students who have been in the district for most of their years in school and those students who had arrived only a year or two before. In addition, comparisons of how girls score on some subtests in contrast to boys could alert educators to issues of gender equity.

Thirdly, the testing could be useful for some particular programs, specifically identifying students for gifted and talented programs and as part of the deliberations in preparing Individual Educational Plans

(IEP) for special education students.

The fourth potential benefit of the achievement testing pertained to its use in assisting individual students by providing practice in taking this type of national test, giving students information on their learning, and demonstrating that other people, in addition to their local teachers, cared about the skills and information assessed on the test.

Finally, the test results could provide useful information for local accountability. They could give 203

Foster citizens one type of information on how their students are doing in comparison to a national sample and on how their tax money was being used.

The potential benefits of achievement testing were / summarized in the Foster school board testing policy which stated in part:

The standardized testing program in Foster schools is designed to provide district personnel useful data on student achievement, ability, aptitude and interest. Standardized examinations yield information that help a Board of Education assess the degree to which its school system is accomplishing stated educational objectives. Test scores provide community members one measure of pupil growth. It is the position of the Foster schools that standardized test data -- properly derived, interpreted, and disseminated — can be a genuine asset to the school district’s educational program.

Potential misuses. Foster educators expressed three overall fears with regard to the potential misuse of the achievement test resultB. The first and primary fear was that the test results might be used to compare teachers, principals, and districts against each other. With what they believed was a mistaken notion that comparisons would build competition and competition would bring about better schools, the state was seen as fostering an unhealthy competition of school districts against each other. Since such comparisons would be based only on narrowly focused test scores, they would ignore all the different factors that influence why some students get high scores and other 204 students get low scores on these types of tests, and would make unfair Judgements about the strengths and weaknesses of educators and schools. Publication of test results by schools and by districts was seen as promoting internal bickering among educators, without helping students at all.

The concern within Foster was that if the test results were organized in a way that permitted comparisons among principals and among teachers, the same unfair conclusions could be drawn about individuals and schools locally as could be drawn about districts throughout the state.

The second concern pertained to whether the test results would be seen by the public as representing a completely valid and relevant measurement of instruction in

Foster, when they were seen by educators as being of limited utility for several reasons:

- The California Achievement Test, although it matched the reading curriculum fairly well, tested "language expression" in a way that was contradictory to the writing process approach to teaching language in Foster. As a result the test results might unfairly represent what students know about language expression. In addition, the math subtest had been compared to the local course of study and found not to be a very good match.

- Any standardized test is only a short snapshot of a student’s work and cannot be taken as representative of what he/she would do at other times, under other circumstances, or on subject matter that does not lend itself to being measured by written, multiple choice, single correct answer, silent, controlled, and timed assessments. A teacher’s observation of the child working provides a more accurate assessment of what a child can and cannot do.

- Foster is moving away from instructing students in terms of the fragmented, isolated skills which 205

standardized tests measure. Instead it is focusing on integrating skills into complex applications by such approaches as the writing process, whole language, literature based reading, application of math skills, and hands-on approach to science. If it becomes essential to have studentB score well on tests of isolated skills, this may drive instruction back to what they see as outmoded and discredited approaches. Thus, in the views of Foster educators, "teaching to the test" implied teaching to a narrower and misdirected conception of what knowledge and skills are worthwhile.

Finally, Foster educators saw this achievement testing as being required for the political purpose of accountability and not for the educational purpose of helping with instruction. As a result, it was important not to base instructional decisions on the test results.

Norm-referenced achievement tests do not yield enough information on any given topic or about the thought processes of any particular student to make sound decisions about students or about instructional approaches from the results. Consequently, in their view, the tests have no value for the ongoing, daily instruction of students.

Actual use of the test results. Once the tests were administered and the results became available, the research focus shifted to how the achievement tests were actually used in Foster. In general, the local scoring, analysis, and reporting process worked well. The locally designed report format which was sent to parents, as well as those created for teachers and principals, were described as much better than those available from the publisher: clearer, 206 easier to read, less threatening.

Within the schools, although having the scores available before the summer was better than what the test publisher had provided, there was still some concern with regard to the usefulness of the results "with only 17 days left to school". It was unlikely that the results would be

Bent in any summary form to the next year’s teacher, although a copy did go into each child's cumulative folder so the individual scores were available if the next teacher chose to look at them.

Each principal was given the results for each classroom and a grade level summary for his/her building. Different principals forwarded the test information in different ways. Some gave the results to each teacher individually; most talked to groups of teachers by grade levels; and some handed them on to teachers without comment. In general, the test results were discussed by grade level to see if there were any surprises or any questions that could be raised about instructional practices.

In terms of particular students, the achievement test results were seen as most valuable in combination with the

"ability" resultB. By comparing the two, teachers could decide if they were sufficiently challenging each child.

Also, by looking at the item analysis which reports how

each child scored on each set of skills, it would be possible for next year's teachers to group students for 207 instruction on those skills.

In one school where the results for a particular subtest were uncharacteristically low, an additional analysis was run to see how the students in the school had done on each item in that subtest. It was discovered that over one-fourth of the items in the subtest dealt with a skill the students had not been taught prior to the testing. Deliberations among teachers and principal in the building focused on whether to rearrange the instructional scheduling in the future so that students would be instructed on those skills prior to the administration of the California Achievement Test in April, rather than afterwards, in May, as currently taught. In addition, the principal planned to take a closer look at the test results over the summer.

In general, the test results were not shared with students, so students did not receive this information about their own performance. Older students have not seen the test results from previous years which may have contributed to the indifference which teachers reported

some students displayed during the administration of the

tests.

With regard to particular programs, there was some

questioning of the wisdom of testing the entire third grade

in order to select a very small proportion for the gifted

and talented program. Although the special education 208 program might use the tests for making some placement decisions, more refined information on the child was usually available elsewhere. The emerging "Instructional

Assistance Teams" might find these test results helpful, yet teachers explained that they did not need these test scores to know which studentB were struggling; they had ample evidence from daily interactions with students.

As noted earlier, of particular concern to Foster educators was the possibility that teachers and schools would be compared to one another based on these test scores. This outcome was avoided this year by giving each principal the scores for only his/her building, and by giving each teacher the results for only his/her classroom and the grade level for the school as a whole. (At the middle school, the results were given to teachers by teams, e.g., sixth grade, team 1, team 2, etc., rather than by classrooms.) So each teacher had the relevant information for his/her students, but not information about the students of other teachers, except for the school as a whole. Each principal had the results for his/her building, but not for other buildings. (A separate report was prepared for the school board which provided totals by grade levels for the district.)

Avoiding these comparisons will be difficult, if not impossible, next year when the state testing law requiring that test scores be reported to the public by school goes 209

into effect. Currently Foster administrators are described by teachers as being "savvy" about the limitations of these tests and do not use them to evaluate the effectiveness of teachers. Results by classroom are kept confidential. In the future, outside pressure to put more emphasis on the test results may undermine this approach.

With the exception of the one school noted above, no additional analyses or special reports were generated after

the initial findings were presented. Principals were pleased with the reporting formats available for the end of

the 1988-89 school year, but made no requests for additional reports organized around the new class rosters

for the 1989-90 year.

Outside the schools, the tests results were sent to

each parent, and a summary report for the district was presented to the school board by the assistant

superintendent in mid-June, 1989. The parent report was a more simplified version than that previously supplied by

the test publisher. The locally designed report provided

the child’s national percentile scores for each of the

subtests of the California Achievement Tests. The

"cognitive skills index" from the Test of Cognitive Skills

was not included in the parent report because it is

commonly mistaken for an IQ score.

In general few questions were received from parents

about the test results, with the exception of the one 210 school which had lower scores on the one subtest. The majority of the questions which did come from parents pertained to a confusion over "percent" and "percentile" scores. Parents were mistakenly reading the percentile score as a "percent correct" score, b o rather than understanding a 60th percentile to mean that their child scored as well or better than 60 percent of the students taking that test, they understood it to mean their child had only answered 60 percent of the items correctly.

Teachers and administrators spent most of their discussion time with parents clarifying this common misunderstanding about the interpretation of norm-referenced test results.

In his presentation to the board, the assistant superintendent used bar graphs to display just how far the

Foster results were above the national norms. In general the audience and the board seemed to feel reassured that

"all was well" regarding the test results. Although there were two questions during the presentation regarding whether these results would be expected for suburban schools and whether there was a possibility of obtaining suburban norms for future comparisons, there were no further questions about the test results from board members or the superintendent in the days or weekB after the presentation.

In summary, the Foster educators used the year prior to the state testing mandate to develop their internal 211 capacity to deal with the achievement/ability teat data.

They acquired the hardware and software to process the test results themselves and they administeredr scored, analyzed and reported the findings within a month. In addition, they were able to forward the test information to principals, teachers, parents, and the school board without making what they perceived as unfair comparisons among teachers and schools.

Furthermore, these test results were not an issue for discussion at an end of the year retreat for administrators. Given the reservations about the relevance of the tests to the curriculum and instruction within

Foster, the lack of questions from parents and board members and the lack of attention given the findings by administrators may be reassuring. On the other hand, given the fact that over 1800 students took the tests, and that the testing absorbs considerable resources in the district, it could be seen as ironic that the results were not worthy of more attention.

The rightful place of achievement testing in the district is still being debated in Foster. Some educators are grateful it is not administered before the third grade and would like to see it eliminated altogether at the elementary level. OtherB would like to see it administered every year so that student profiles could be developed from test results. As noted in Chapter I, the state mandate to 212 give publisher-prepared norm referenced achievement tests

is seen in Foster as contradicting another state mandate to develop competency tests aligned with the local curriculum.

Some educators see Foster as needing to develop its own test which would authentically measure its local written curriculum and approach to instruction. Such a test would

serve as an appropriate challenge to both students and

teachers. Such test development is time consuming and

costly, however, and may not be a priority when more

immediate demands are pressing in on teachers and

administrators.

Theme 2: Effect of Testing on Relationships

In general, as noted earlier, interview results and

observations of interactions among teachers and

administrators provided a picture of Foster schools as

characterized by supportive and professional working

relationships among most educators in the district. There

seemed to be a deep respect for each other’s capabilities

and each person’s commitment to students. The research

question reported here pertains to what effect an external

state testing mandate may have on these organizational

relationships. The findings are reported in terms of

relationships among educators, relationships between

teachers and students, and relationships between educators

and the local community. 213

Relationships among educators. Given that many administrators have serious doubts about the benefit of the proficiency testing and some concerns about the merits of the achievement/ability testing, the state mandate puts them in the position of requiring teachers to do something that both they and the teachers find hard to justify. Since the testing often goes against the professional judgement of both, it puts administrators in the position of being seen by teachers as "agents of the state", enforcing and

"selling" a mandate that administrators themselves have serious concerns about. This position can be detrimental to relationships built on respect for each others* professional judgement.

In addition, educators spoke of the care, time and energy that has been put into developing Foster's courses of study. Teachers are committed to the local curriculum and take very seriously the expectation to complete its contents by the end of the year. The time pulled away to give the achievement/ability testing is seen by teachers as seriously intruding on the teaching time available in the spring. Teachers are then caught between two contradictory instructions from their administrative leaders: give this achievement test and complete the instruction called for in the course of study. This tension can affect the working relationships between administrators and teachers. 214

Prior to the administration of the achievement test there was a feeling of "unknown" regarding whether the test results would end up reflecting badly on teachers. Although there was no stated intention to Judge or compare teachers based on student scores, there was uncertainty expressed by some teachers over whether this might happen. This uncertainty put additional stress on working relationships.

Once the achievement scores were distributed, both principals and teachers reported that the scores were not used to evaluate or compare teachers. Uncertainty still remains, however, for future years when the state requirement to report the scores by school will add emphasis to the test results.

Furthermore, some administrators report their role as one of encouraging each teacher to develop his/her leadership in different areas in the school. They fear that the state testing mandate, rather than emphasize and build upon diversity of skills, encourages all teachers and all students to be Judged by the same criteria. In this way it works against the individual development of teachers which local administrators would like to enhance.

Although teachers report knowing that standardized test results are useful only in combination with many other sources of information, they feel that if there is pressure felt from administrators to do well on these tests, this pressure may get passed on to each other in terms of who is 215 to "blame" if students do not do well. In such a climate it would be e a s y for a teacher to assert that students were not taught well the year before.

In addition, there is currently an interest and commitment to working on interdisciplinary applications of knowledge and skills among teachers in the middle schools.

As a result, time is taken away from teaching one’s own subject in an isolated manner in order to coordinate it with the teaching of other subjects. Although teachers and administrators feel this may be the best approach to teaching higher order skills and complex application of knowledge, it might result in Btudents scoring less well on a test of isolated content and skills. If this testing turns out to reflect badly on teachers, the tendency could be to pull away from cooperative, integrated instruction and go back to teaching alone. Cooperative teaching relations could suffer as a result.

For administrators, there is a concern that next year's state requirement to report results by schools could also adversely affect the relationships among building principals. Combined with the state legislature's new

"choice" legislation which will allow parents to choose their child's school, the test results could easily be used by parents as a shorthand, if inaccurate, measure of the quality of education in each school. Principals will then be put in the position of competing with each other on the 216 basis of test scores. Such an outcome is seen as detrimental to the quality of education in each building.

Relationships between teachers and BtudentB. Although teachers believe that learning takes place through many repeated cycles of introduction and repetition, there was a concern that results from the state proficiency test could lead to the conclusion that if the students do not know it, they were never taught it. Teachers feared that state legislators may not understand the difference between students being taught something and retaining it later for a test.

The concern over the upcoming proficiency testing is that teachers might start to place undue stress on specific factual content in order to get students to retain isolated bits of knowledge for the state test. As a result of pressure to have students do well on the test, teachers may become more "test-oriented" than "student-oriented". They may look for all students, despite individual differences, to gain a particular score on a onetime measure, rather than look for each student to show growth over time in a variety of areas, a focus more in keeping with the ideal of educating individuals.

Furthermore, there are many activities which are beneficial to the development of students, especially middle Bchool students, that will not be measured on the state proficiency test or the achievement/ability tests. 217

The "advisor-advisee" activities at the middle schools are examples of time used in the middle school to help students cope with adolescence and develop self-esteem and competence in a variety of areas. These activities are seen as helping to develop relationships among teachers and studentB which in turn enhance the academic as well as emotional and social growth of students. Yet if great emphasis is placed on test results, teachers may conclude that they cannot afford to take time for activities that do not directly teach to the tests.

The standardized testing conditions themselves can affect teacher-student relationships. Although teachers do try to treat the testing matter-of-factly in order not to create undue stress, the administering and taking of the tests is a tiring and tedious process, one that both teachers and students must submit to. The strict restriction over what teacherB can and cannot say in the testing situation sometimes puts teachers in a position of not being able to give students clear answers to their questions and helpful guidance regarding how to approach an item. In addition, teachers cannot read these test questions aloud as they often do with classroom tests.

Students who have learned to expect clarifying help from their teachers cannot always find it on standardized test days. In the view of Foster educators, these circumstances can affect the trusting relationship established between 218 teachers and students.

Relationships between educators and local community.

The state testing mandate requires that schools give particular norm-referenced achievement tests, ability tests, and a state-developed proficiency test. The assumption in the local community could be that because the state requires them, these tests must be a valid measure of what local schools are supposed to be teaching. If students do not score well on the tests, the local educators are in the position of having to explain why to community members who are well aware of the taxes they pay in support of schools.

If in fact the tests are not a good match with what is being taught and the instructional approaches used in the schools, educators, particularly administrators, must explain their alternative approach to curriculum and instruction in a compelling and non-defensive way. The vast amount of detail that is necessary to explain why some children and some subject matters are not fairly assessed with these types of tests has to be offered in resistance to the tendency of the general public to want only summary information in abbreviated form.

Foster educators feel that explanations about particular subtests or particular items that contradict the district1s approach to instruction need to be heard so that it is not wrongly concluded that students are missing 219 particular skills or content. For example, the mismatch between what is referred to on the California Achievement

Test as "language expression" and what is taught as a part of Foster's writing process curriculum could generate the false conclusion that "grammar" is not being taught in

Foster. In fact, educators assert that students are learning grammar as part of a comprehensive approach to learning how to write, so students must do the more difficult task of incorporating grammatical rules in their daily writing, rather than the easier but less justifiable task assessed on the test of making a multiple choice selection of a 'noun1 or 'capitalization error’ from a string of words or a sentence that someone else has written.

Making these complicated arguments to board members and other members of the local community is necessitated by the fact of state mandated testing. If the community does not understand why these tests are not an accurate reflection of what teachers are teaching and students are learning, the subsequent strain in relations may cause educators to acquiesce to teaching to the test, even if they disagree about the benefits of such an approach for educating students. In effect, if relationships with community members demand that they do well on the tests, even if doing well on the tests is not in the best interest of educating children, local educators are hard pressed to do 220 otherwise.

In summary, Foster educators' feared that the state testing policy could affect their work and their relationships with each other and with students and community members. As described above in connection with their professional dilemma, the concern was that pressure

from the state could filter to the local community, to the school board, to the central office administrators, to principals, to teachers, and to students in such a way that educators start to do what is best for how they will be perceived by those above them in the hierarchy, rather than do what they think is best for students both now and in the

long run. The fear was that because no one wants to be perceived as unsuccessful in their work, a potential undue emphasis on these test results as the measure of success could cause individual educators to start basing their administrative and instructional decisions on what will make them seem successful, rather than on what they think

is best for students.

Theme 3: Benefit of Fall vs. Soring Achievement Testing

During the first phase of data collection for the

inquiry (January to April, 1989) there was considerable concern among Foster educators regarding whether the achievement/ability testing should be done in the fall or

spring. The state testing policy left the choice of when to 221 test sixth and eighth graders up to the local district. The contrasting opinions are summarized below in terms of the benefits associated with each choice. (This issue was not raised by those interviewed after the testing was completed during the second phase of the data collection, May 1989 to

March 1990) .

Benefits of fall testing. Some administrators and teachers believed that information from the California

Achievement Test regarding students' mastery or non-mastery of specific objectives could be used for instructional decisions by current or subsequent teachers if testing was done in the fall. They asserted that any hope that the testing would be useful for an instructional purpose (as opposed to a political purpose of comparing Foster with other school districts) would be lost if testing was not done in the fall.

Proponents of fall testing also noted that students would be getting ready to get out of school in the spring, so fall testing would be a better time to assess their achievement.

Benefits of spring testing. There was a widespread belief that test scores from spring testing would be higher and thus more favorable. Some educators asserted that if

Foster tested in the fall, its scores could be unfairly compared with neighboring districts who tested in the spring. (This widely held opinion seemed to be based on a 222 misunderstanding regarding how fall and spring norms are used. Foster students tested in the fall would be compared to the publisher's norming group tested in the fall to derive Foster percentile scores; those tested in the spring would be compared to the publisher's norming group tested in the spring for their percentile Bcores. A subsequent comparison of percentile scores across districts would not give an advantage to either the fall or spring tested students.)

Another concern was that fall testing would catch students before they had overcome learning loss from the summer; before they had regained study habits; when they were not used to being in situations similar to those imposed by the test; and when their learning was closer to where it had been the year before, rather than where it would be at the end of the current year. (For example, in the fall new sixth grade students are more like fifth graders than sixth graders). It was believed that in the spring students would have mastered new skills; would be more in the "school mode"; and would do less guessing.

In addition, spring testing would be useful for program evaluation purposes. Since there was not enough information from the test to make wise instructional decisions regarding individual students, it would be best to use the results for help with planning the following year's program. The information could be useful for teachers to 223

see if students learned what teachers thought they had been

teaching.

Finally, fall administration of the the California

Achievement Test would interfere with the eventual fall

administration of competency tests which would be more

aligned with the local courses of study. Foster educators

hoped to soon develop criterion-referenced competency tests

which would measure accomplishment of the local curriculum.

Since these tests would be useful for instructional

purposes, that iB, would provide teachers with the

information they need to plan instructional interventions

for pupils, the competency tests need to be given in the

fall. Furthermore, it was felt that since it was important

to have the teaching focused on the local courses of study, not on the content of the CAT, fall testing of the local

courses of study would make more instructional sense.

Theme 4: Effect of Testing on Local Curriculum

The findings related to this fourth theme are first presented for the proficiency and achievement/ability tests

separately and then the effects of both forms of state mandated testing on the local curriculum are considered and summarized together.

Proficiency testing. As described previously, the state mandated proficiency test consists of ninth and twelfth grade assessments in four areas: reading, writing, mathematics and citizenship. For each area, the state has

specified and published "learning outcomes" on which

students will be tested. These outcomes aerve to both guide

the development of the test (to be done by a commercial

test developer under contract to the state) and,

potentially, to guide curriculum and instruction in local

schools. In its Spring 1989 edition of its Teacher Update

newsletter, the Ohio Department of Education suggested that

these learning outcomes be shared with parents and

community members, be discussed by the total instructional

staff, especially middle and high school teachers and

counselors, and be used to "inform students about specific

areas addressed in the ninth-grade learning outcomes that

they will be expected to know" (p. 4). Discussion with

students who were in the seventh grade during the 1988-89

school year was especially encouraged, since these students would be the first to feel the impact of the proficiency

testing policy.

In the fall of 1990, all ninth grade students will take the ninth grade proficiency test; those who do not pass it may take it again before the end of March 1991. These students must pass the ninth grade test in order to receive a "diploma of basic competency" when they graduate in

1993-94. They will continue to receive two chances each year to take and pass the test. If they still do not pass, they will receive a "certificate of attendance" upon 225 completion of their high school curriculum. To receive a higher level diploma (one with distinction or commendation), these students must also pass the twelfth grade proficiency test. In addition, during March of the

1990-91 school year all tenth, eleventh and twelfth graders will take the ninth grade proficiency test.

As described earlier, an initial Foster’s response to the proficiency testing policy was a district-wide meeting called by the assistant superintendent. He invited interested eighth grade and high school English, reading, mathematics and social studies teachers to a discussion of the proficiency testing mandate in early February 1989.

Accompanying the invitation was a list of the state learning outcomes and an indication of where, if at all, each outcome could be found in the Foster courses of study for reading, writing, mathematics and social studies. As a consequence of this meeting, eighth grade teachers at the middle schools and all English, math and social studies teachers at the high school were asked to complete a form for each of their courses indicating whether a particular learning outcome from the state proficiency test was taught or not taught in each course. These formB were turned into the assistant superintendent in late February, 1989.

The intent of this data collection was to see exactly in which courses which students were being instructed on the content and skills to be assessed by the proficiency 226 tests. After some initial confusion over whether all high school teachers of these subjects (even those who do not teach ninth and twelfth graders) should complete the forms, and whether it should be done by individuals or departments, it was concluded that to be accurate the information would have to be collected from each teacher for each course. This process was envisioned as a first step to deciding where Foster curriculum and instruction would need to be changed in order to "ensure our students preparedness for these tests" (from a 1-11-89 memo from assistant superintendent).

A considerable "mound of forms" was collected and as of

March 1990 has not been analyzed. The task is seen as enormous and the number of people available in the central office to work on such items has been curtailed. Budget cuts have resulted in some central office administrators working in classrooms part time as of the 1989-90 school year. In addition, given that principals and teachers opened the year with over 600 more students and without hiring additional staff, the assistant superintendent felt he could not ask educators in the schools to help with this task.

The research question of whether the state's proficiency testing mandate would affect the local curriculum elicited six areas of concern from educators interviewed and observed at meetings during the period of 227

January 1989 to March 1990.

First, the question of how the proficiency testing would he beneficial to students was described as a non-discussable issue* That is, since the state had already mandated that the testing be done, the question for local educators became how to respond to the mandate, not whether and in what way the test may benefit or hurt students.

Local educators expressed that they felt silenced on this important educational issue.

Secondly, the question of will they "teach to the test" and, if so, how, was a central issue. As described earlier, the concern over whether the local curriculum and instructional practices (including the timing of when particular content is taught) would be revised to accommodate the state mandate has been a focus of much discussion in the district. If such accommodation takes place, does it makes most sense to introduce a "crash course" on the content included in the state' s learning outcomes just prior to the test administration or to revise the local courses of study to embed these outcomes at the grade levelB required by the state proficiency test? It was felt that a crash course approach might work for the math and citizenship outcomes which require a review of factual information and specific mathematical operations taught previously or the introduction of some new content. Such a course would probably not be useful for the writing and 228

reading outcomes, however, which deal with more complex

skills that have to be developed over a long time.

Allowing the state proficiency test to drive the local

curriculum was seen as contradictory to the earlier state mandate for local districts to develop their own courses of

study in the areas of reading, English composition and mathematics. As noted above, by the 1989-90 year, districts were to have developed competency assessments to see if

students were accomplishing the pupil performance

objectives included in the local courses of study. By mandating state proficiency testing in these same subject areas local educators believed that state legislators were replacing state wide learning outcomes for those developed

in the local courses of study and were dictating when in the curriculum certain content be taught (that is, prior to the administration of the proficiency test).

A particular problem for Foster educators related to the state proficiency outcomes in the citizenship area.

Some of the content on the proficiency test is taught in

Foster as early as fifth grade (and is subject to being forgotten by the time of a ninth grade test) and other content is not currently introduced until the eleventh and twelfth grades (too late for the state ninth grade testing). Accommodating the state's learning outcomes for citizenship would require a major shift in Foster’s eighth grade social Btudies curriculum taught in the middle 229 schools.

Furthermore, time spent teaching toward the state’s learning outcomes was Been as time lost toward the teaching of the local courses of study. Teachers were already concerned over not having enough time in the year to effectively instruct a curriculum to which they were committed. Diverting instructional time toward the state tested items, which may be seen as inappropriate to the developmental level of their students at that point in time, added to the quandary for teachers who took seriously their professional responsibility to teach the local curriculum.

The third concern focused on the question of whether a state-wide mandate describing the learning outcomes for all ninth and twelfth graders could be beneficial to students in over 600 different Ohio school districts. This question raised the accompanying issues of whether a standardized curriculum was in the best interest of all Ohio students and whether legislators should be the ones to influence the content of the courses. Foster educators were concerned that teaching to a state test could mean not serving the needs of local students.

The fourth concern was related to one raised in connection with the effect of testing on relationships with students. The learning outcomes of the proficiency test might be better suited to the content-oriented emphasis of 230 the high school years than the student-oriented emphasis of the middle school years. If the fact of the state test encouraged middle school teachers to become more content oriented, they might have to give up the time currently spent on developing self-esteem and social skills of students, as well as time spent on developing personal student-teacher relationships which they believe enhance the instruction of academic content. Within the middle school, there is the belief that if teachers and students spend time getting to know each other as individuals it is more likely that teachers can reach studentB academically than if they are strangers. Teachers feared that an overemphasis on covering content because it will be on the state test could result in not developing such relationships which foster student academic growth in the long run.

In addition, as discussed above, the time consuming activities that can be beneficial for the overall development of the adolescent (e.g., advisor-advisee sessions, assemblies, activity sessions based on student interests) are not measured on the state tests. If teachers become test oriented, they could not afford to focus on these non-measured aspects of student development.

Furthermore, a middle school teacher can currently offer individual students a wide variety of ways to express their knowledge. To complete an English assignment, for 231 example, a student, may be offered the possibilities of writing a paper, reciting a poem, or completing some artwork. In the testing situation, only one form of expressing knowledge is allowed. Those who are not good

"test takers" have no other avenue in which to demonstrate their competence. In addition, middle school teachers of

English, math and social studies might feel less flexible than their science teaching counterparts because of the state testing emphasis on specific content in these areas.

The fifth concern centered on what would happen to individual students and to the curriculum if students do not pass the ninth grade proficiency test. Will new courses be developed for them? Will they take eighth grade courses all over again? Will they have to have new approaches developed for them? What learning opportunities will they forego while they are preparing to retake the proficiency test? The answers to each of these questions would impact the local curriculum.

The final concern regarding the proficiency test pertained to the differentiated diplomas. The four tiered systems starts with a "certificate of attendance" for those students who fulfill their local high school requirements but do not pass the ninth grade state proficiency test, and ends with a "diploma of commendation" for those who pass the twelfth grade proficiency test and meet other criteria established by the Btate. Teachers and administrators are 232 concerned that although such a system may help to focus young people’s attention more on education, with legislators defining what this education should be through proficiency testing, the differentiated diplomas may do more harm than good. It was felt that the type of a diploma a student received would not make a difference to legislators, but it would make a large difference to students and their future life chances. Such diplomas could further serve to label students and thereby limit their possibilities in life.

During the interviews conducted in March and April 1989 many Foster educators expressed the view that they were not overly concerned about the state proficiency test forcing a change in their curriculum. They had a great deal of confidence in the strength of their current courses of study and their instructional program and they were not anticipating the effect of the proficiency testing by making changes now in their eighth grade curriculum. If several years of test results, however, were to show students consistently scoring poorly in a particular area, educators stated they would then reevaluate their curriculum in that area. But as of Spring 1989, they did not feel such a reconsideration was needed in light of the confidence they had in the local curriculum.

Starting in mid-May 1989, however, the mathematics department at the high school began a serious 233 reconsideration of its curriculum and instructional practices, including its division of students into four

"tracks" for mathematics courses. In combination with the new standards promulgated by the National Council of

Teachers of Mathematics, and with a review of a University of Chicago mathematics program, the teachers became engaged in a dramatic revision of their orientation to the teaching of mathematics. A university course running through the

1989-90 school year was developed as a vehicle for Foster teachers to plan their new mathematics curriculum. The mathematics learning outcomes for the state proficiency test were included in these deliberations.

Among the ideas being considered is the preparation of a local mathematics test to be given at the beginning of the ninth grade year. Such a local test would serve several purposes: a) satisfy the state mandate for a local competency test in mathematics; b) provide practice for students who will take the state proficiency test 6-8 weeks later; and c) provide instructional information for teachers regarding which students need instruction on which mathematical concepts and skills. Currently the state proficiency outcomes are being viewed as a "baseline" content that all Btudents should know. Mathematics teachers continue to deliberate, however, over how much, if at all, this test should dictate their curriculum for the 1990s. 234

Deliberations are also ongoing over the revision of the

Foster language arts and reading courses of study. For the first time these two subject areas are being combined into a single course of Btudy document. As noted earlier, a committee of reading and English faculty began their deliberations in early June, 1989, met throughout the summer and continue to meet during the 1989-90 school year.

The issue of how much the state proficiency learning outcomes and the reading and language items on the

California Achievement Test will be incorporated in this

Foster course of study has been debated constantly during these meetings.

Achievement/Ability Testing. As noted previously, achievement and ability tests were administered in grades four, six and eight in Foster in April 1989. The California

Achievement Tests (CAT) in reading, language, and mathematics and the Test of Cognitive Skills were given in anticipation of the grade levels and subject areas required of the state mandate effective for the following school year. In addition, grade three students were administered

CAT reading and mathematics subtests for local use in selecting students into the gifted and talented program.

Three potential effects of the mandated achievement testing on the local curriculum have been described in terms of other themes: 1) the fear that achievement tests which do not match their local courses of study, especially 235 in language and mathematics, will result in a change in the local written curriculum; 2) the fear that such testing could drive their instructional practices backward toward the teaching of workbook pages and skills in isolation, rather than focusing on the the application of complex skills; and 3) valuable teaching time would be diverted to testing and away from instruction.

Two additional concerns were raised in connection with the achievement testing. Some teachers wondered if the timing of the testing dates could make a difference in student scores and indirectly affect the curriculum. For example, prior to the testing week in April, one group of middle school students had just completed a several week interdisciplinary unit during which the combined efforts of the math, English, social studies and science teachers allowed them to study a topic in depth. Coupled with spring vacation, it had been several weeks since the students had had the more traditional instruction of subjects in isolation. Since the CAT assesses not only subjects but skills in isolation, students may have been at a disadvantage given their lack of recent exposure to the particular topics and format of the test. The future scheduling of interdisciplinary units in Foster middle schools therefore could be influenced by the administration of the state mandated achievement testing in the spring. 236

Furthermore, student scores could be affected by the disruption of scheduling routines for the testing. In both the elementary and middle school buildings, normal class schedules had to be altered to accommodate the timing requirements of the tests. Such disruptions may affect student performance. In addition, some teachers expressed discomfort with the role of "test giver", either because of unfamiliarity with the requirements of test administration, and/or because they disagree with the purpose, format, or content of the assessment. Teachers' attitudes about the test could also affect student performance.

One influence of the state testing policy was evident during this first trial year of implementation in Foster,

Despite strong predictions from some school principals, most teachers interviewed did not foresee an immediate change in the local curriculum due to achievement test

results. That is, those interviewed prior to the point in the study when the actual achievement test scores were distributed declared they would not teach to the test.

Instead they suggested that if Btudents were to score poorly on particular test items which did match their local curriculum, this outcome would cause teachers to look at their instruction. If, on the other hand, students scored poorly on items that were not well matched with their local curriculum, they predicted that teachers would not change the curriculum or their instructional practices as a result 237 of the test scores. Given their strong belief in the value of the local curriculum, teachers and some principals did not anticipate changing it due to achievement test results.

That is, they did not foresee themselves teaching to the test, especially since they maintained multiple choice tests do not reveal the student's thinking process, and do not provide accurate information about individual students from which teachers can base instruction.

After the test results were returned and the students in one of the schools performed poorly on one subtest, there was considerable consternation from parents about the results. The principal spent several days looking over an additional item analysis (provided at some time, energy and material expense by the central office) in order to uncover which items caused the problem. It was discovered that the content of 13 of the 50 items on that particular subtest had not yet been taught by teachers who were following the timing sequence in the local written curriculum. It was therefore legitimate for students to not know how to approach these specific test items. It was interesting to note, however, that students in other buildings throughout the district did well on these same items.

The interpretation given this peculiar outcome from veteran administrators in the district was that teachers familiar with the test content from previous years do in fact insure that students have instruction on that 238 particular content prior to the state testing, even if the instruction would normally come after the testing dates if teachers strictly followed the sequence of the local curriculum. In this one building, however, there were many new teachers who had not given this same test in the past, and the "informal" knowledge about the content of the test was not available to them. Teachers in this particular building played it straight, teaching the local curriculum in the sequence they thought it was supposed to be taught.

The conclusion that can be drawn from this episode is that this form of testing does affect what gets taught in the classroom, whether or not the content is included prior to the testing or at all in the local curriculum. Teachers who know what is on the test from past experience insure that its content and the format in which the content is presented is taught in their classrooms. Whether this is an unconscious choice, or a conscious but unspoken choice, the overall consequence is that the test does influence the

"lived" curriculum in the classroom. This is a subtle, covert influence, however, one that does not come to the surface unless something goes "wrong".

Argyris & Schon (1978) refer to this distinction as the difference between espoused theory and theory-in-use. The espoused theory of these educators was that they believed in the local curriculum and used it, not the content and skills of the state mandated test, to guide their 239 instructional practices. Their theory-in-use, however, takes into account the high political stakes associated with state testing, and insures that their students have received instruction on the content included in the test prior to the test administration. Since they as teachers, as well as their students, their school, and their district could "look bad" if the scores are low, the curriculum and instruction are altered to insure that students have every chance to score highly on the test.

Both proficiency and achievement/ability tests.

Finally, there were two additional issues raised by educators for both sets of state mandated tests. As will be described in Theme 5, there was an overall concern that such testing pulled the scarce resources of time, money, energy, and attention away from instruction and directed them toward testing, without the tests offering anything valuable toward instruction. In this way, the state testing program could be seen as contributing to the failure of students, rather than providing resources toward their success.

Secondly, educators could be caught in a "Catch-22” dilemma regarding the two sets of state mandated teBts. If teachers push hard on the isolated skills which are measured in the achievement tests they would not work on student development of the higher order skills and writing competence measured in portions of the proficiency test. If 240 teachers prepared students to do the more complex tasks measured in some parts of the proficiency test, students could score poorly on the isolated assessments of the achievement tests. In either case, it was felt that educators would be criticized for not preparing students to score well on the tests.

To summarize the main findings for theme four, the impact of the testing policy on local curriculum, both the proficiency and achievement testing were seen as having the potential to change the written courses of study and the instructional practices used in Foster schools. Despite strong statements that they did not want this to happen, and from some educators, that it would not happen, there is already evidence that the California Achievement Test is altering the "lived curriculum" in the classrooms of teachers who have administered it in previous years and are thereby familiar with its content. Given their professional commitment to their locally developed courses of study,

Foster educators want this, and not the state testing provisions, to guide their instructional practices. They are vulnerable, however, to the negative consequences of having students score less well than might be expected in their school system, so their instructional practices are changing, if subtly or unconsciously, to insure that students have practice in the types of content and skills assessed on the state tests. 241

Theme 5: Effect of Testing on Local Resources

The 1987 Ohio legislation creating this state testing policy required school districts to bear the costs of the achievement/ability testing although state funds were to be used to develop, administer and score the proficiency tests. This section of the findings summarizes the Foster resources used for the achievement/ability testing of grades 4, 6, and 8 and the achievement testing of grade 3

(approximately 1800 students) in spring 1989 and the preparation done to date for the state proficiency tests.

In this discussion, "resources" include both the time, energy and attention expended by personnel and the non-personnel monies expended on the testing process.

Achievement/Ability testing. As noted earlier, Foster’s response to the state mandate to test the achievement and ability of students was to make the testing process amenable to local needs by investing in the hardware and software to score and analyze the tests themselves. By taking local control over the scoring and analysis process they were able to to give the results back to principals, teachers and parents within a month of completing the testing and before the end of the school year (a service not available from the publisher) and are now capable of doing repeated analyses of the scores in order to answer long range local questions about the performance of 242 students on the tests. Such answers are not available when purchasing standard scoring services from test publishers.

Capital (building) funds were used to establish this in-house capacity, with the understanding that this would save the money ordinarily spent on purchasing scoring services from the operating (general) fund in the future.

In addition to the monies invested in purchasing hardware and software, time, energy and attention were needed by central office personnel to select vendors, and to develop the local capacity to score and analyze the tests. Even without the decision to do local scoring and analysis, district resources were needed to select the particular achievement and ability tests to be used in

Foster, purchase and administer the tests, get them scored and analyzed, and deal with the results. A detailed list of the precise tasks performed by Foster educators, secretarial staff and students for this testing process is provided in Appendix B.

There were over 50 specific, time consuming tasks completed by the assistant superintendent, other central office administrators, principals, teachers, secretaries and students to carry out the achievement and ability testing. These tasks included such substantive activities as preparing and presenting training sessions for teachers, administering the tests, answering parent questions, and presenting the findings to the school board. They also 243 included many logistical tasks such as counting out test materials, erasing stray marks from answer sheets, and filling envelopes with individual student reports for parents. The list in Appendix B specifies the exact tasks done by each type of personnel in the district and by students. It provides an overall view of the consumption of human resources required by this form of testing.

In addition to the time, energy and attention of its educators, support staff and students, the achievement and ability testing consumed Foster financial resources. The total non-personnel costs for the 1988-89 year to complete the achievement/ability testing process was $48,800, which included the long term investment in hardware and software enabling the district to score and analyze its own tests.

As noted in Table 3, this figure also included testing materials and supplies, royalty fees for publishers’ norms and the prediction formula (used to predict achievement scores from ability scores, which is one of the reporting requirements of the state testing mandate).

In subsequent years, the district anticipates Bpending an estimate of $3,000-4,000 for the non-reusable test answer sheets and test booklets for grade 3 (which does not use answer sheets). Without local capacity for scoring and analysis, the annual testing costs were estimated at

$6,000- 8,000. The internal scoring ability thus saves

$3,000 to $4,000 each year to apply toward the initial cost 244 of software and/or hardware. In addition, the hardware can be used for purposes other than test scoring and analysis, and the district now has the capacity to score different types of tests such as those which will be developed to comply with the state requirement to do periodic competency testing in reading, writing and mathematics. Such testing would have also required the purchasing of scoring services. Furthermore, as noted earlier, with the hardware and software investment, the district now has the capacity to control the timeliness of the scoring and the relevance of the reports generated.

Table 3: Foster Expenditures for Achievement and Ability Testing. 1988-89

Hardware $28,000 (Building/capital fund) Software: $12,000 (Building/capital fund) Testing materials & supplies $ 6,000 (General/operating fund) Royalty costs for norms: $ 2,400 (Building/capital fund-one time cost) Royalty costs for achievement/ ability prediction formula $ 400 (General/operating fund-ongoing cost estimated at 0.20/pupil)

Total for 1988-89: $48,800 245

Proficiency testing. Although state funds will pay for

the actual administration and scoring of the proficiency

tests, local resources have been expended in preparation

for the testing. Foster resources of time, energy, attention, and paper have been expended on the following

tasks:

February, 1989:

- Prepare for and conduct meeting to discuss proficiency test (22 Foster eighth grade and high school teachers and central office staff attended for 1 hour)

- Small group meetings within two middle schools and high school to explain and complete form identifying which proficiency test learning outcomes are taught in each course by English, reading, math, and social studies teachers.

Ongoing, May 1989 to March 1990:

- Periodic meetings of high school and middle school math teachers to reconsider approach to teaching mathematics in high school, including deliberations regarding how to incorporate proficiency testing learning outcomes.

- Reading and language arts course of study revision committee meetings included deliberations on proficiency testing outcomes for both reading and writing.

- Social studies middle and high school teachers have deliberated on several occasions regarding how to best prepare students for citizenship portion of state proficiency test.

Purposes and consequences. As part of their conversations regarding local resources spent on achievement/ability and proficiency testing, Foster 246 educators repeatedly raised the issue of the purpose of the testing. On the one hand, it was clear that the state mandate provided the impetus for the district to consider how it could use achievement testing for both state and local accountability, as well as to take control over the scoring and analysis process. There was also some belief that the achievement testing could provide useful information for evaluation of the instructional program.

On the other hand, there was the concern that this form of testing does not assess what they are teaching and thus did not provide useful or valid information. Even worse, the test results could make teachers look bad for not teaching isolated skills, when they do not believe that teaching skills in isolation is best for educating students. In addition, the test does not provide enough information on particular topics or skills and the thought processes of individual students to be helpful for tailoring instruction for those students. As a broad measure of which students are doing well and which ones are not, the test results were seen as redundant to what teachers already know. The testing was then characterized as just an exercise students are put through; one that does not benefit their learning. As such, it was seen as a waste of valuable resources.

Foster educators also noted that the state mandate places a financial burden on all districts, but especially 247

on those which do not have resources to absorb it. The

proficiency testing especially was seen as unlikely to

provide new information to high socioeconomic districts,

and thus was considered a waste of both state and local

resources.

Furthermore, the district resources of time, energy,

attention and money were needed for the development of

assessments which do match their local courses of study.

Foster efforts to respond to the ongoing state mandate to

develop competency assessments in reading, writing, and mathematics to match the local courses of study and to administer these assessments at least once in grades 1-4,

5-8 and 9-11 had been put on hold while the district

responded to the new mandate to give norm-referenced achievement/ability tests and have 9th and 12th graders

ready to take the state developed proficiency tests.

Consequently local resources have been pulled away from

assessments which were seen as relevant and useful to local

instruction and devoted to testing which was seen as having

questionable utility.

The issue of scheduling provides a further example of

the paradoxical aspects of this testing. In the middle

schools, the testing week was the only time all year that

the students and teachers had to forego their advisor-advisee activities period. This was seen as an

unspoken yet powerful statement regarding the influence of 248 this type of testing on the "normal" operations of the school. For next year central office administrators are anticipating implementing a standardized testing time across the elementary and middle school buildings, rather than have each school try to adjust its own schedules for the testing. Although this effort helps the standardization process, it works against another effort to make the classroom environment as close to normal as possible so that the students can do their best work on the test. And once again, such a radical change in scheduling draws even more time, energy and attention to the mandated testing and away from teaching.

Another paradoxical issue pertained to the scoring process. As a result of doing the scoring internally, district administrators became acutely aware of the problems associated with quality control over the answer sheets. Such problems as stray marks, incomplete erasures, even students marking items for subtests they did not take, can cause the scanner to read and thus score the tests inaccurately. During April and May 1989, central office personnel spent several days scrutinizing the answer sheets to avoid the computer era dictum of "garbage in, garbage out". In the future, they predict having to bring the issue of quality control over answer sheets to the attention of principals, teachers and pupils to insure the accuracy of the internal scoring process. Paradoxically, drawing more 249

time, energy and attention to the answer sheet pulls even more resources into this form of testing, and away from both teaching and the form of assessment which is seen as a more accurate reflection of what they are teaching. As put by one educator: "Last year at this time, I was wondering how to avoid this testing. This year I'm wondering whether

to hand out pencils so we will have consistent markings for

the scanner,"

Summary of Themes 1-5

To summarize the anticipated effects of the state testing policy on the work and relationships of educators

in Foster in terras of these five themes, the policy was seen as potentially useful as well as potentially harmful, depending on whether the quality of schools, teachers and the district were judged on the basis of the test results.

If such judgements and the ensuing comparisons were to be made, the testing policy would be viewed as harmful to the education of students. Furthermore, such testing could adversely affect professional relationships among educators and between educators and students, and between educators and the local community.

If educators viewed the norm-referenced achievement test results as useful for instructional decisions, they preferred testing to be completed in the fall of the school year. If they viewed this form of testing as having little 250 utility for the instruction of individual students, but of some use for evaluating the overall instructional program, they favored spring administration.

The impact of the state testing policy on the curriculum was being felt for both the achievement testing and the proficiency testing. Teachers were making subtle changes in their classroom practices as a result of the achievement testing and deliberations among groups of teachers have been ongoing regarding how to incorporate the learning objectives specified for the proficiency test in the written courses of study in reading, language, math and social studies. Considerable local resources, both financial and human, have been expended in conducting the achievement and ability testing and in preparing for the proficiency testing. As a result, these resources were not available for what are viewed as more appropriate assessment and accountability effortB.

In sum, the mandated testing represented a troubling state policy for Foster educators. Although accountability was seen as necessary, this form of accountability was seen as a redundant, irrelevant, and in some ways, misleading assessment of Foster's instructional program.

Talking about Testing: Discourse Rules

The previous description of how Foster educators anticipated the effect of the state testing policy on their 251 work and relationships was concerned with the subject matter of their conversations with each other and with me as researcher. In other words, the findings pertaining to the research question, "How do local educators see the policy influencing their work and relationships?" focused on the content of their talk rather than on the discourse process itself. This section of the findings returns to a consideration of the ongoing conversations and addresses the following two research questions:

c. What rules for discourse shape the state testing policy?

d. What rules for discourse shape the professional culture of local educators?

These questions are addressed by analyzing the research findings in three ways: 1) contrasting the discourse rules of the state testing policy with those of the professional culture; 2) examining the implications of the conflicting discourse rules for articulating Foster's vision of education; and 3) examining the implications of the conflicting discourse rules for Foster educators' conversations with local constituents.

In this discussion I am referring to 'discourse rules* as the norms which shape the conversation among educators, including the way in which people talk to each other; the types of conversations that are allowed to take place; the 252 topics that are permitted on the agenda; and who has permission to talk about which subjects in what manner.

Rules for Policy and Rules for Culture

The first analysis focuses on the distinction between the discourse rules implied in the state testing policy and those which represent the norms in the Foster professional culture. The state testing policy implies a set of rules for conversations related to giving and receiving orders and prescriptions. The wording of the policy itself specifies exactly what the State Board of Education, the

State Department of Education and the local school boards shall do. Examples include:

The State Board of Education shall formulate and prescribe standards of a statewide program to assess student achievement in relation to student ability...

The State Board shall annually designate the date or dates on which the tests prescribed under this section shall be administered...

The Department of Education shall adopt rules prescribing the manner in which each test prescribed...shall be administered to students and properly graded and scored...

The Board of Education of each city...shall administer and provide for the scoring of the test prescribed... on such dates and in such a manner as prescribed by the rules of the Department. (Ohio Revised Code 3301.077-078) (emphasis added)

The "just do it" aspect of the state policy troubled

Foster educators who were accustomed to being consulted 253 about matters pertaining to their professional work. As noted in an earlier quotation from a building administrator: "You can't force things on people. You can't force things on professional, educated people. You just can’t do it. It has to evolve over time, and they have to accept the change and want to do it."

In contrast to the "forcing on people" implied in the state policy, one Foster administrator described how they usually approach a new development facing their organizational reality:

This staff is very receptive to new ideas, they are very dedicated, they're very loyal, and they’re very hard working people. So, to get them to do things is really not a hard task, but it’B not easy either. You have to do it in a very professional way and approach them as professionals...When something comeB down the pipe new - we definitely solicit the input of the staff: "What do you think of this? How would you like to do it? Give us some of your ideas". And we all, ultimately, make the decisions.

The assistant superintendent was operating according to the professional cultural norm of shared decision making when he called together middle school and high school teachers to discuss how they might respond as a district to the state proficiency test. As described earlier, he knew that they had to do something to prepare students for the proficiency test, but he preferred to develop the district’s course of action through collaboration with teachers. The first meeting regarding this policy included the following statement from the assistant superintendent 254

about their conversation: "I am so fortunate that I can

talk to you about this stuff; other administrators [in

other districts] can't dare have this type of conversation with teachers,"

At a subsequent meeting a few weeks later, he asked the

department head for high school mathematics: "Can I problem

solve with the group for a minute?" A year later at a

February 1990 meeting of social studies teachers, he was

still asking if they should do anything different with the

curriculum to prepare the current eighth graders for the

ninth grade proficiency test to be given in the fall.

These cultural norms for participation in decisions and

consensus building are strong, yet they cannot overcome the

fact that the state tests will be given regardless of

educators view the tests. At the first meeting in which

they were reviewing the state learning outcomes for the proficiency tests, a number of teachers noted that "some of

this isn’t appropriate at this grade level". A few minutes

later a teacher asked: "What is the long term worth of this

for the child?" Another teacher responded with irony: "That

isn’t an issue, obviously". In other words, the worth of

this test for students k b b not an item they could put on

the agenda for conversation. As noted by the assistant

superintendent: "It’s a done deal, a fait accompli". The

best they as educators could do was decide if and how they

wanted to change their local curricular and instructional 255 practices to improve students’ chances of scoring well on the state test.

During this same initial meeting in early February

1989, there were many simultaneous conversations among the

22 teachers present. Although the assistant superintendent and the curriculum coordinator were eager to turn the conversation to the logistical task at hand of developing a way to determine where and when each of the state learning outcomes was currently taught in their instructional program, the teachers were deeply engaged in comparing the state learning outcomes with their written courses of study and with dealing with what they termed their "deep philosophical problem" of whether they should teach to the test. It seemed that despite the efforts of the meeting conveners to finish the task at hand so as not to impose on any more of the teachers’ voluntary time at this after-school meeting, the teachers themselves needed to verbally process the information regarding the state proficiency test and the implications such a policy has for their teaching.

One of the indicators of an organization’s culture is the metaphors used to describe it. Mumby (1988) asserted that metaphors are more than "stylistic embellishments of literal language, but actually shape the experience of social actors" (p. 18). He goes on to note that "military metaphors can characterize highly structured, formal 256 organizations* while family and organism metaphors might exemplify more flexible systems" (p. 18).

Foster administrators repeatedly invoked metaphorical images when they described the state testing policy and its effect on them. One administrator proclaimed:

It's just like being in the military: "Go do it" "Okay, yes sir. I'll do it." This isn’t the military. And the state doesn't control education in Ohio.

Another principal recalled being asked by a group of teachers, "Why are we doing this [achievement testing]? His ironic reply, "Because I said so, that's why" was taken as intended, that is, as a not-to-be-questioned direct order which stood in dramatic contrast to their customary way of conversing about educational issues.

Foster educators also invoked industrial or factory metaphors when talking of the lack of fit between the policy makers' views of schooling and their own. Such terms as "product", "raw materials", "bottom line", "business",

"productivity", and "competition" sprinkled their talk as they described their perception of how policy makers view schooling. Administrators argued against the assumptions embedded in a view of education which implied that all schools start with the same "raw materials" (students) who can be "processed" on an assembly line consisting of teachers stamping out subject matter knowledge in equal 257 amounts to each child. They protested that the productivity of the school could not be measured by judging the quality of the "product" (the child) and expressing this judgement in numerical terms. Examples of their metaphorical language include:

Principal: [The state testing reflects) a factory mentality, that everyone has the same raw material... everyone starts in the same place, everyone ends at the same place, with the same product. And that’s not what we have. So the tests don’t match the children and the children don’t match the test and so ...we spend time teaching to the test or we let the test define us...What then becomes the rule of thumb is not what’s best for the children, but what’s best to make us 'look good’ in the factory line.

Principal: I think these people are looking [at education] as a business, [looking for] a product, and we don’t have a product. We don't see a visible product when they leave here..We hope that they have learned their skills and we get surveys back [telling us]., that they are doing OK, but we don't see that product.

Principal: I view the legislature...as a bunch of businessmen, by and large, who use industry’s format of productivity based on competition. And the way we are going to measure productivity is some statistical means [like] statewide testing... And it really shows a lack of foresight on the legislature’s part of what education is all about in this country.

Now obviously there iB a difference there, because when [business] wants to produce more [it’s] going to measure its productivity in dollars. You can’t do that with education. You can’t measure our productivity in dollars. So what iB the legislature going to come up with that’s close to dollars? Well it's numbers. To anyone who is an educator, in my opinion, who has really been out in the field and haB worked with kids and has worked with teachers, worked with the community, those numbers don’t mean a whole lot to people. 258

Central office administrator: What they [policymakers] want is a number. That's what they want is a number. And that also is a gross misconception of where we are with children's development. Children are not numbers. We are not a product oriented business. We are not a business. We are learning centers for people. People are not numbers. And it'B grossly misrepresentations! to even attempt to reduce anyone to a number.

These administrators resisted policy makers’ focus on numbers, as represented in this instance by test scores.

The tendency to talk about numbers instead of about people is another distinctions in the discourse between the policy and the professional culture.

A final example of the clash between these two ways of talking is provided by a principal:

But the end product [of education] will never be measured by a test score. It'll be measured by something well beyond that.

...And if it could be measured by a test score then you don’t need people like me here...You could have a computer do it, give them a source of instruction on how to take the test and how to have the information in the format the test wanted. If you want, we could set that up and you are wasting a lot of money on me and on a lot of other people.

...But my job is much more important than that. That is such a narrow part of the job, that it’s not worth me spending a great deal of time on, so consequently I don't worry about that. I ’m worried as much about how do the children feel about themselves and knowing that that’s going to affect their productivity more than the fact that on a particular day, on a particular test, they were able to process this information.

...That's a world apart. Those are light years apart in expectations. And what we need to do is get away from only being able to articulate the most rudimentary part of our task at hand here, and take people to those other levels. That's the challenge of being an educator 259

in the 2000s. We have to get to the point of being able to articulate those other areas.

...One of the problems in articulating that is that the system doesn't want to hear those kinds of things. They want strictly objective data. They love numbers. We are going to have to be able to articulate the subjective data that we currently have that gives us a reason for being what we are today as an educational syBtem.

...But a lot of the [educational] leaders that we have don't have experience in doing that. Number one, they don't have the experience of being with kids to even know how to communicate that and number two, they are driven by a system that doesn't want to hear that anyway, even if they did know it.

...People almost make fun of you if you were to talk about that,

In summary, the discourse rules implied in the state testing policy focus on giving orders, with a "just do it" military assumption that the orders will be followed without disagreement or negotiation. Furthermore, the policy implies that test results can and probably should be used in conversations to compare the quality of schools and school districts. In contrast, the discourse rules of the professional culture include the assumptions that professional decisions regarding curriculum and instruction will be made in collaboration among teachers and administrators, and that changes in past decisions will be made only after consultation and consensus building.

Educators resist thinking of schooling in terms of the metaphorical images of factories and armies, and focus on people, both students and teachers, instead of numbers, which they feel could never adequately represent the 260 complexity of the teaching/learning process.

Articulating a Vision of Education

As noted above in the distinction between policymakers * focus on numbers and professional educators’ focus on people, Foster administrators expressed frustration with the inability of their profession to articulate the value of the rich "subjective" information that they have about student learning. This lack of ability to voice an alternative form of accountability is especially difficult for administrators who are in the position of having to communicate all the time. They are the intermediaries between the public and the teachers and they are the spokespersons for the schooling process. Although some writing is required, their primary discourse is oral and they are required to do it all day long. Yet they have not been able to articulate in a convincing way how the schooling process should be authentically judged and how they and teachers should be held accountable for the public trust and the public funds. Examples of this frustration from Foster administrators include:

Central office administrator: We are our own worst enemy. We don’t do a good job of articulating exactly what it is that we do and how we do it to the public... So we get back to that magic word accountability; we have to be accountable for what we are doing. I guess everybody has to to a certain extent, This [testing] will placate some of those people who are saying, are those kids really learning? And they can point to the test score and say yes they are, or no, they’re not. 261

Principal: With state wide testing we in a sense sold our souls to the legislature to let them do whatever they want to with,..And I think that one of the reasons that education throughout the country is in what kind of shape it is in, it’s because we have been too Boft with people, and we have been very non-assuming, and sit back and let people do whatever they want to with u b .

And we are not good at telling people, "NO" we're not going to do that and these are the reasons that we are not going to do that. And take us to the wall, but don’t expect us to do something that we know is not good for kids.

Whether or not the Foster educators can find a way to

say "No" to the negative implications of the state testing

policy and find a way to articulate an alternative form of

accountability to their local community is the focus of

Story 3. The final analysis addresses the implications of

the two contrasting set of discourse rules for the

conversations between educators and the public as

represented by the local school board.

,Conversations with Constituents

Given that some Foster educators disagree with the premise of the state testing policy, namely, that the

quality of children and the worth of the schooling process can be reduced to a test score, how do they talk with local policy makers and community members about the test results?

One portion of the 1987 state testing legislation required that the test scores be published "in a Btatewide report 262

for all school districts and the general public that compares student achievement scores among districts in

relation to student school and mental ability" (RC

3301.078H). Subsequent 1989 legislation required that the

scores be published by district and by school so that

comparisons across schools within a district and across

districts are virtually guaranteed.

If Foster educators could follow the norms for their

practice within the professional culture they would first

establish that the tests matched the content and approaches

used in their teaching. They would then view the test

scores as one of many indicators of a child's academic

performance, and use the child’s score to confirm or

disconfirm other data. Similar to the practice of a doctor

when diagnosing and treating a patient, the teacher would

combine different sources of information to make

instructional decisions about the student.

Building administrators could use the test results to

confirm or disconfirm other indicators regarding the

different subject areas being taught in the school. If

students scored poorly on a subtest, administrators could

ask if students had been taught the assessed skills or

concepts. If what they know about the teaching practices

and the results of other indicators of student performance

indicate that students should be able to answer the test’s

questions, administrators could look more closely to see 263

why students did not score well. In essence, the testing

would be used "in-house" for individual and program

evaluation, as one of many indicators to check if students

are learning what teachers are teaching.

In contrast, the state testing policy implies that the

test results will be used to compare schools and school

districts. Within Foster the fear is that the results will also be used to compare teachers. Coupled with the belief

that the tests do not accurately assess what they are

teaching, and that children come to them with different needs and different likelihoods of success on these tests,

Foster educators feel such comparisons are not only unfair

to them as individuals but are dysfunctional to the

educational process as a whole.

In the following passage the assistant superintendent talked of the conflicting uses of the test results and how he attempted to deal with the conflict:

My preference is to provide the director of elementary education, for example, with building by building profiles, and have that individual charged with the responsibility to review that data and to say there's an anomaly here and I need to figure out what it is. But the way in which I want to do that is by carefully reviewing the data, talking with the principal, working with the principal and observing a few classes without getting it out in the newspaper and having parents trying to decide whether they’re going to move because [of the scores of a building]...I want to avoid that, almost at any cost. Because I think that it’s destructive.

...We're trying to say to principals, this has utility within these bounds, and stressing this at 264

every point. We want to foster in the instructional leaders the attitude that this is just a snapshot in a big portfolio of student work. And if there's something that’s really out of sync in this third grade teacher's room, and you didn’t expect it...Then you have a right to pursue it. It's not a big stick, it’s just a red flag.

.,.Not any differently than what we say to a classroom teacher. We expect that this will confirm what you already know. If it doesn’t, it's something you want to probe...If there’s discontinuity there, then you need to verify it. You're the professional after all. So that’s the attitude that we're trying to nourish across the district. It's going to be very challenging. I don't see it done successfully in most places.

...I think what is essential iB that you provide the service to teachers so that you make the testing as easy as possible. And that you provide them leadership in saying we understand what the uses of this test can be. We are supportive in it not being used to judge your teaching performance or to make decisions about promotion or retention of students. It may be one piece of information but it should not be pivotal to exhibit that kind of understanding.

This disagreement over the proper use of the test results constrains the discourse between administrators and the local board. Since the spring 1989 administration of the achievement test was done one year prior to the effective date of the Btate mandate, Foster was not obliged to publicly report their scores building by building.

Avoiding the dissemination of this information was important to the administrators in order to avoid what they viewed as unfair comparisons. Consequently, they had to be very careful in their presentation of the findings in order 265 to give accurate information about the results without inviting school by school or teacher by teacher comparisons.

In mid-June 1989 the assistant superintendent presented the achievement test results at the public school board meeting. His 35 minute presentation included a description of the Board’s testing policy, the different uses made of different forms of norm-referenced and criterion-referenced tests, cautions about the differences in the content and approach between this test and their local curriculum and instruction, and charts and graphs noting the percentile and stanine scores for each of the four grade levels tested on each of the subtests. No building by building information was reported, and the emphasis was on how well the district did in comparison to national norms.

Two aspects of the discourse between the assistant superintendent and the board are worthy of note. First, although longer than the 10 minutes the Board initially allotted, the 35 minutes used were not enough to clearly present all the background knowledge that was needed to accurately interpret the test data. Consequently, the presentation was done at a fast clip, used technical jargon, included many "asides" which were meant to explain

important aspects of testing but which only confused the naive listener, and moved quickly past the two questions

that were asked by a board member. In effect, the 266 presentation served to reassure Board members that "all was well" in terms of their district’s standing in the nation, but did not educate them about the strengths and weaknesses of the information they were receiving. It served the function of avoiding comparisons within the district but did not set the scene for changing the way the schools are held accountable in the district.

Secondly, the emphasis on how well the district did in comparison to the national norms ignores the fact that suburban children usually perform well on such nationally normed tests. The two questions asked of the assistant superintendent during the presentation pertained to the issue of whether a comparison with other suburban districts might be more informative:

Board member: "One question.... how does this compare to the surrounding school districts?

Assistant superintendent: "I haven't had a chance to look at that and part of the problem doing that is that districts do testing at different grade levels and we've got districts using [different testsJ...We 1ve got one district using CAT and we can get some of that, but I've not had a chance to deal with that.

[Several minutes later:] Assistant Superintendent: What does it mean for Foster? Look at some of the stanine distributions. Grade 3. The darker band represents Foster students. I can give you some more current information on this. In Grade 3, 75% of our students scored in the 6th stanine or above...in reading. In math, 72% of the students taking the test scored in the 6th stanine or above. So you see the performance is really very positive. Now I didn't include, there are some other tests, word analysis and study skills, but in the interest of time I haven’t provided any overheads on that. This then is third grade, aggregated together, look at the class of 540 267

students and they are scoring 76th in reading and 79th percentile in mathematics, in grade 3.

Yes?.

Board Member: Is it fairly common, that suburban schools, as opposed to the national average..

Assistant Superintendent: There is such an animal known as a suburban norm that we don't currently have and that we have sought for the system and over time as we develop some sophistication and have some resources to get the job done we can develop what you are asking for, suburban norms.

Board member: I mean wouldn’t you sort of expect suburban school to have somewhat higher..

Assistant Superintendent: I think this is probably consistent with our expectation level and one of the other significant pieces of information that we are currently missing is the so called discrepancy between ability (I put that in quotes, I don't believe personally that any test can show you that) between student ability and achievement. That is the information that we will have to report to the state beginning in the fall. We just joined a number of school districts in pressuring McGraw Hill to release the technical information so we can derive those scores locally. In fact, I ’ve been told...that we will be able to get that information and save the district [some money] in the process.

This conversation was noteworthy for its failure to deal with the issue being raised by the board member, namely, that the test scores being presented may not be

entirely reflective of the schooling process in Foster

since one would expect affluent suburban Btudents to perform this well. In his concern not to emphasize thiB

testing any more than necessary and to control the agenda of his presentation in order to avoid being asked to compare schools within the district, the assistant 268

superintendent basically avoided the underlying question

being asked.

Earlier in the presentation he effectively precluded

questions being asked of him by announcing:

Now I will tell you that I am not a testing and evaluation specialist and I'm sure there are members of the audience tonight who could ask me questions and I couldn't answer them. I wouldn’t make a pretense of being an expert. Testing has become very sophisticated, very statistical. It’s in a field called psychometrics.

His efforts to avoid being asked questions was not

without cause. As noted by another administrator in the

district:

I know [the assistant superintendent] sweated over the presentation to the board last night because he’s already had questions earlier in the year. Because there were some board members who would say, "How did this class do at this school?"...What they want to end up doing is saying this is a better teacher than that one. That was his major fear because that question had been raised by that particular board member at another time so that's why he worried about it so much.

I later asked the assistant superintendent if he had

received any further questions about the test results or had been pressed to make comparisons. The following dialogue expresses his intentions regarding the discourse with the School Board:

Assistant Superintendent: We have [avoided school and classroom comparisons] and that's without having had a chance to do any reality checks on the presentation. Which I may not have pursued anyway with the board 269

members because I don't want to get into an area that takes a great deal of time to explore, to explain, to understand... And it just opens up all kinds of potential problems I didn't want to deal with anyway.

Researcher: So you don’t even want to raise the topic of conversation

Assistant Superintendent: I do not.

Researcher: unless they do?

Assistant Superintendent: And they didn’t. And in that regard I consider it a major success.

Researcher: Yes.

Assistant Superintendent: My objective was to present information that didn't evoke any controversy. To try to show Foster in the good light that I think it deserves based on student performance, and to do it in such a way as to demonstrate that we acquired hardware and software, we processed information in a very rapid time, got it back into our client’s hands (in this case, teachers and parents as well as building administrators)...without a major hitch. And we have some pretty good information which profiles student performance on a test that I don’t think ought to be overinterpreted.

To the degree that I can minimize my time on this I think we are being effective, because as we have discussed already on several occasions, the more you commit your time to this via discussion, response, the more you emphasize the test. We don't want to do that.

Controlling the amount of time and attention he devoted to talking about test results which were seen to be of limited value was one way for this administrator to minimize the way the discourse rules for the Btate policy shape his work life. Although he had stated previously that the Foster curriculum was not so idiosyncratic that it cannot be partially assessed by the state required

l 270 achievement test, he would rather devote his conversations

to the forms of assessment and accountability measures which would more appropriately reflect what Foster

educators are doing for students in Foster schools. In addition, he resisted the implied discourse rule of the state policy which encouraged him to compare teachers, schools, and districts on the basis of the test scores.

Given that the rules for discourse of the state policy conflict with the rules in the professional culture, it appears to be difficult for the assistant superintendent and the school board member to have authentic conversations about testing. Establishing new rules for such conversations is thus an important first step in the process of articulating Foster educators’ vision of schooling and in developing a method of accountability which both the educators and their constituents would see as legitimate.

Summary of Story 1

The state testing policy represents an external

"entity" constructed by actors outside of the organizational reality of the Foster B c h o o l district which must be interpreted and acted upon by its members. How it impacts their socially constructed organizational reality depends upon how they make sense of it and what actions they take in response to it. 271

In sum, the findings reported in Story 1 reveal that

Foster educators' initial reaction to the state testing policy was one of ambivalence: they could see the need for accountability measures but resisted the state specified achievement and proficiency tests as appropriate means for assessing their written curriculum and instructional practices. In addition, the policy created a professional dilemma for them: they could either act in accordance with their professional beliefs about what education should be for Foster students or they could "teach to the test", which represented teaching to a narrower and misdirected conception of what knowledge and skills are worthwhile, but which would make themselves, their students, and their school "look good" to those who judged them on the basis of test scores.

In terms of the policy’s effect on their work and relationships, Foster educators are already manifesting a subtle change in their instructional practices based on the high stakes associated with the achievement testing.

Whether and how the learning outcomes for the state proficiency test will be incorporated in their written curriculum is still being deliberated by reading/language, mathematics, and social studies teachers. The policy could have an adverse effect on professional relationships once the publication of the results by schools and the comparisons which come from such reporting are made. This 272 form of testing consumes considerable district resources, both human and financial, rendering these resources unavailable for the development of what would be viewed as more accurate and appropriate assessments of the local instructional program.

Moreover, conversations about the testing policy were constrained because of the different discourse rules implied in the policy and in the Poster professional culture. In contrast to the cultural norms for collaborative and participative decision-making on educational issues, the testing policy implied that administrators should just command teachers to give the achievement/ability and proficiency tests. In addition, the scores from these tests were to be used to make comparisons across schools and districts in the state. Within the professional culture, the preferred use of the test scores would be to first insure that the assessments matched the instructional program, and then to combine the scores with other sources of information to help guide instructional decisions for individual students and programmatic decisions for buildings and schools. In other words, the test scores would serve the internal information needs of the professional educators in their deliberations about practice,

Given the order to administer state determined tests and to report scores to the local community, coupled with 273 the current climate of criticism of the schools and strife among board members and an exploding pupil population,

Foster administrators were having difficulty articulating their vision of education and the reasons that the state tests did not adequately measure how they currently educate pupils. In order to avoid making the comparisons implied in the policy, the conversations between administrators and board members were abbreviated and strained. They did not have open discussions of the test results during which administrators could offer non-defensive explanations of the lack of fit between the state tests and the Foster curriculum and instruction.

Whether Foster educators and board members can engage in new conversations about their schools could be adequately assessed and how educators could be legitimately held accountable is the focus of Story 3 in this chapter.

The role of the research process in the Foster conversations about testing is addressed next in Story 2.

Story 2: How Did the Research Process Relate to the Conversations in the Professional Culture?

The findings presented in Story 2 concern the interaction between the research process and the professional context of administrators and teachers within

Foster. This section describes how the research methods and 274

the person of the researcher affected and were affected by

the conversations about the state testing policy, in Story

2 the primary focus is on the individual conversations

between the researcher and the persons interviewed as well

as between the researcher and the key informant. Story 3

focuses on how the research process might influence the

collective conversations in Foster. The following specific

research questions are addressed in Story 2:

a. What rules for discourse shape the research process in this organization?

b. How does the researcher influence the individual conversations about the state testing policy?

c. How is the researcher influenced by conversations with the educators in this organization?

After providing some background on myself as researcher, the research methods used, and the conversational settings in which the research was discussed, this section analyzes the rules for discourse inherent in the research paradigm, with a detailed look at how these rules were manifested in this study. It then describes the way the research process influenced individual Foster participants. The final discussion of how the relationship between the key informant and myself evolved over the course of the study provides an example of how this type of inquiry can unfold to influence both the researcher and the researched. 275

Background of Researcher

As a former teacher in the Chicago Public Schools and evaluator of educational programs in Wisconsin and Rhode

Island I have always been struck by the complexity of the teaching/learning process. Trying to understand this complexity in order to teach and later to describe it accurately for the programs 1 was evaluating has always been a personal and professional challenge.

In the midst of my first year of teaching I was enrolled in a Master’s degree program and its obligatory research course. I have a keen memory of sitting in the

DePaul University library trying to control the impulse to throw the bound edition of a research article out the window into the cold and gray Chicago air. In all my undergraduate and graduate work 1 had never had to force myself to read the subject matter of the courses. It always had held some intrinsic interest, that is, until 1 encountered this article and others like it. The reported research was so removed from any student, teacher, or administrator it was hard to imagine it was conducted in a school. Its description of the isolation and manipulation of operationally defined variables by a detached, value free researcher was so foreign to the daily life of students and teachers in neighborhoods scarred by poverty and race riots that 1 could not believe it could inform my practice. I concluded that if this article represented 276 educational research, I wanted no part of it.

1 later learned 1 was alienated from the positivist paradigm of research and its requirements for the inquirer to remained detached from the object of study in order not to contaminate it and for the complexity of life to be parceled into isolated pieces and controlled in order to be studied. Upon learning of the interpretive paradigm of research which acknowledges the impossibility of separating the knower from the known and which seeks understanding of human social life in all of the complexity of its natural context, 1 felt 1 had come home. Alternative research approaches, variously called naturalistic, interpretive, collaborative, participatory, emancipatory, ethnography, and action science, seemed to better fit the school environment and to hold the possibility of informing practice. Such approaches could serve the actors in the schools, not just the career interests of external researchers. In discovery an alternative manner in which to inquire about the complexity of schooling, research became for me a supportive, informing process. In Ross Mooney's words:

...such that research then becomes a way of generating life and sharing of its nutriment, serving, then, the primal aim of education in the affairs of men, i.e., Bharing life-in-mind as means to life in men. (1975, p. 174) 277

With the possibility of research generating and nourishing life in schools, serving educators because it is done for them, not just about them (Carr &. kemmis, 1986), 1 undertook to find a research site for an as yet unfocused dissertation, 1 sought:

- a place where teachers and administrators talked with each other in a manner that allowed/encouraged self and collective reflection; where the norms for the talk "allowed in" topics of teaching, learning, and struggling with new ideas;

- a place where I could study the relationships among educators without the topic seeming threatening;

- a place where 1 could watch the effects of state policy on the daily work of administrators, and indirectly, of teachers.

- a place that would let me "shadow" one or more administrators;

- a place where an ethnographic, qualitative, participant-observation approach to research would be accepted as ordinary and appropriate and 1 could openly admit to my perspective and world-view without pretending to be detached from the issues and events i was witnessing;

- a place where the research 1 would do might somehow benefit participants, and not be another occasion of local educators being used by university researchers.

My doctoral course work had focused on the question of how policy, organizations, and language relate. The works of Blumer (1969), Bateson (1972, 1979), Berger & Luckmann

(1966), Goffman (1971), Hall (1977), and Wilden (1987) alerted me to Bee organizations as essentially people in various relationships and discourse as the vehicle for creating, maintaining, and changing those relationships. As 278 a result* 1 was interested in focusing on the discourse of members in an organization* especially the discourse used to enact an externally initiated policy.

Although I knew 1 wanted to study the impact of a state policy on the practice of educators, 1 had not yet chosen a specific policy to study. During our first conversation,when we were exploring the possibility of my doing research in the district, the Assistant

Superintendent for Curriculum and Instruction in Foster recommended to me the state testing policy. He was concerned about the effect of this policy on the district's professional culture which he had been working to establish and maintain. Since this particular policy was expected to have a significant impact on schools, and i had some background on the technical aspects of testing and its use in program evaluation, i happily accepted his suggestion to shadow him in his efforts to implement this policy in

Foster.

Research Process and Conversational Settings

As described in Chapter III, my research strategies included observing Foster's naturally occurring conversations regarding the state testing policy and initiating such conversations by asking to interview individual administrators and teachers about the policy.

Further occasions for conversation were the "reflection 279

meetings", as I referred to them, between the assistant

superintendent and myself. As my key informant, Bill was my

access to others in the district, the chief supplier of

background information on questions and issues 1 raised,

and the person with whom I could discuss the theoretical as

well as practical implications of the state testing policy

in Foster. We had 13 reflection meetings between March,

1989 and March 1990, typically lasting about an hour.

In addition to issues I raised from my interpretations

of the data collected and analyzed to data, our

conversations evolved to include discussions of the

possibility of my teaching a university course in the district focused on assessment and accountability issues.

We also decided to jointly write an academic journal

article in which we each described our perspectives on the

development of our researcher-researched relationship. Bill

proposed the joint writing project and he identified and

named the phases through which he saw our relationship

evolve. The particular features of the evolution are described in a later portion of this story.

Another research-initiated occasion for conversation

occurred when I sent back early drafts of my writing. As noted above, in May 1989 I sent the 15 educators

interviewed to date a list of the six themes I had

identified and a six-page summary of the first two {use of

the test results and effect of testing on relationships). 280

In a cover letter 1 asked for their feedback in terms of specific questions about the accuracy of the summary. I also asked if my directing questions to them about the state testing policy or their reading of the summary itself might influence their subsequent view of the policy. Five of the 15 people responded by phone or in writing and I initiated another interview with two others in order to acquire more direct feedback.

Subsequent drafts of the remaining themes were given to selected readers, and in February, 1990 all 26 people interviewed received a complete version of the findings to date. This version included the data collected regarding the six themes subsequent to the initial May 1989 analysis, and provided an introductory description of the overall

Foster community and of the professional culture for teachers and administrators.

The remainder of Story 2 focuses on how conducting the research influenced some of the conversations during the period of time when Foster educators were initially preparing for both the state mandated achievement/ability and proficiency testing as well as the actual administration of and use of the results from the achievement/ability testing in Spring 1989. It begins with a discussion of the discourse rules inherent in interpretive research and how these rules were enacted during the inquiry in Foster. The impact of the interviews, 281 reflection meetings, and initial May 1989 member check on the Foster participants is then described. Story 2 concludes with an analysis of how conducting the study affected the key informant and myself.

<• Discourse Rules in Interpretive Research

Lincoln and Guba (1985) proposed five axioms of the

"naturalist paradigm", their preferred title for an alternative to the positivist paradigm. Defining axioms as

"the set of undemonstrated (and undemonstrable) 'basic beliefs’ accepted by convention or established by practice as the building blocks of some conceptual or theoretical structure or system" (p. 250), they provided a set of basic beliefs which guide the researcher in carrying out naturalistic inquiry.

Table 4 provides Lincoln and Guba’a five axioms and my attempt to identify the implications for discourse inherent in conducting naturalistic research. In other words, I have tried to identify what the axioms mean for the conversations occurring within the inquiry.

If we accept the first naturalistic paradigm proposition that social realities are multiple, holistic and constructed by the actors in a social system, then research about that system must focus on the constructions of local actors. 282

Table 4. Naturalist Axioms and Discourse Implications

Axioms About: Naturalist Paradigm: Discourse Implications

The nature Realities are Social realities are of reality multiple, constructed through constructed and language. holistic.

The Knower and known Researcher helps shape relationship are interactive, and is shaped lay of the inseparable. conversations being knower to studied. the known

The Only time- and Research focuses on how possibility context-bound the discourse rules for of working hypotheses the particular setting generaliz (idiographic shape the conversations ation statements) are of the members. possible.

The All entities are in Through conversations possibility a state of mutual participants mediate of causal simultaneous and mutually shape linkages shaping so that it external "entities". is impossible to distinguish causes from effects.

The role of Inquiry is Through conversation values. value-bound. researcher*s and members* values shape the way in which data are constructed.

(Lincoln and Guba, 1985, p. 37) 283

Focusing on the language actors use to create meaning is one of the ways to understand their construction of reality. But the researcher cannot do this from a distance.

As Heron <1981) asserted:

The researcher who participates in the overt dialogue of the organizational members in their natural setting also

influences that dialogue, because, according to the second axiom, the knower and the known are inseparable. It is not possible for researcher and "researched" to interact without influencing one another.

The third axiom focuses on the aim of inquiry, namely the development of a "working hypothesis" that describes an individual case. Within the individual case are the discourse rules which shape the conversations of members of particular settings. A part of describing and understanding the specific case includes describing and understanding their agreed-upon rules for talking to one another.

In their fourth axiom, instead of cause and effect linearity, Lincoln and Guba asserted that entities are in

"a state of mutual and simultaneous shaping". Rather than denying that any entities cause others to behave in certain ways, I see this axiom as arguing that causation does not 284

occur only in one direction. In other words, humans "act

back" on the forces that seek to impact them. Human beings

use language to respond to these forces, to develop ways to

deal with the forces and thereby shape their own world.

Finally the value-bound property of inquiry is rooted

in the person of the researcher, who is inseparable from

the knowledge being constructed, influencing that knowledge with his/her own values. These values are expressed in the choice of the question being addressed, the choice of the paradigm guiding the investigation, and the choice of the

substantive theory guiding the specific collection, analysis, and interpretation of the data. Furthermore, the values of the participants being studied influence the unfolding inquiry (Lincoln & Guba, 1985, p. 38). By watching the discourse, the researcher can uncover how his/her own values and those of the members' of the specific setting are constructing the data of the inquiry.

Discourse Rules for Research in Foster

In accordance with Axioms 1 and 4 (multiple, constructed realities and role of causality), I focused on how Foster members constructed the social reality of their response to the state testing mandate by listening for how they made sense of it through their conversations. The state testing policy is one social reality constructed in conversation by state legislators outside of the context of 285

Foster. As such, it is an external "entity" to the local actors, and some would say it "causes" Foster educators to act in a certain way. But the state policy enters Foster’s

socially constructed reality through the sense making and

interpretation of the actors there. It is enacted in combination with many other realities that impact their work, including the norms of their professional culture.

Foster educators create the "structures and arrangements",

in Thayer’s <1988) terms, of their social world through

their conversations. Thus, the research focused on how these practitioners incorporated the policy into the norms of their professional culture.

Axiom 3 maintains that the aim of inquiry is to generate hypotheses about a particular time- and context- bound case. To the extent that other settings are like this one in terms of the features of their professional culture and the introduction of the state testing policy at a time of dramatic and traumatic growth, some of what occurs in

Foster may be transferable to another setting. The focus of this inquiry, however, is upon this policy and this place in this time. It seeks to understand how these educators operating under these particular norms responded to this particular state policy.

Axioms 2 and 5 acknowledge that the inquiry is influenced by the methods and values of the researcher. As a result, a part of conducting the research includes 286 assessing how the researcher's methods and values affect the construction of the data.

In short, the discourse rules which shaped this research process included:

- the researcher interacted with and joined in the dialogue of the participants;

- the researcher revealed (disclosed) her perspective and herself during the conduct of the inquiry;

- through conversations, the researcher influenced those researched; and

- through conversations, the researcher was influenced by those researched.

The following summary explains how these discourse rules were enacted in the conduct of the study.

My general discourse strategy was not to speak at the naturally occurring meetings in which members of the Foster professional culture were called together to discuss the state testing policy. If 3 were asked a question, or if I had some information that was pertinent to the discussion and unknown by others in the group (e.g., the specific dates of the proficiency testing or the names of the levels of the diplomas) I would offer this information. If I was engaged in a conversation, I would not attempt to hide my ideas or reactions to the topics they were discussing. I just did not introduce them unless asked.

During the interviews, my discourse pattern was to introduce the general topic of the state testing policy and 287

its possible influence on the work and relationships of

educators, and then let the respondent take the

conversation in any direction he/she desired. The opening

question was similar to the following quote:

VJhat I ’m interested in, very generally, is how you see the state testing policy influencing your work as a principal, if at all...

Although I initially asked to speak to people for about

a half-hour, most interviews lasted at least an hour and

some approached two hours. During the course of the

conversation, I asked clarifying questions, or offered

examples of what I thought the person was saying as I way

to check my understanding. As these conversations went on

for an hour or more I began to speak more. I asked more

questions or elaborated on an idea the other person brought

up. In general I tried to let the person I was interviewing

reveal their ideas and values early on. Then, if the conversation kept going, 1 revealed more of my values by

elaborating on particular points he or she raised or by bringing in additional issues and asking for reactions.

I particularly listened for the words they used to describe the policy and their reaction to it. As noted

earlier, I was alert to their descriptions of the professional dilemma created for them by the policy and to

the metaphors they used to capture the difference between

their perspective on education and that attributed to state 288 policymakers. In most instances respondents raised Borne or all of the Bix themes described in Story 1. If, during the course of the conversations, a person did not speak to the issue of how the state testing policy affected their professional relationships, I would introduce a question to that effect. At the end of the interviews I tried to summarize what I thought were the main points as an on-the-spot member check.

In most cases the interviews were audiotaped with the understanding that the respondent could push the off button on the recorder (which I identified and put within reach) at any time. No one ever objected to being recorded nor turned off the tape during an interview.

An example of both my attempt to check my understanding of what the other person was saying by rephrasing it in the form of a question as well as my self-disclosure occurred in the following exchange toward the end of a one-and-a-half hour interview. Just prior to this exchange, the administrator was describing how their locally designed parent report form was so much clearer than the one provided by the testing company in the past, and yet parents did not understand how to interpret the one score that was provided to them, the percentile.

Administrator: Doesn't it scare you? If that’s the best we can give them, the most they can understand, they still don’t understand that. 289

Researcher: So the only thing we can give them, that they can understand, is percentiles, and they don't understand percentiles?

Administrator: Uh-hum.

Researcher: Yeah, that is scary.

Administrator: So take that to the next step for people who don’t have that emotional investment in their own children for looking at those test scores. When it keeps going farther and farther away from where it [education] happens, it is only a number.

It is only a number to the legislators who are saying 'This is the score that [a neighboring district]] got; or that [another district] got; and this is the score that Foster got.'

And then what happens to those inner city schools in [names several districts]. What are their numbers going to look like?

Researcher: There’s no way. Or if they bring those numbers up, what have they done to the children in order to get those numbers up.

l’ve been trying to work out in my head some kind of a good analogy. And the closest I’ve come to so far, and it’s not working yet, is to say something to the effect of: "What if your charge is to grow a garden, and the garden had to have as many different colors and as many different times of blossom as possible, the ideal perennial garden."

Administrator: Uh-hum.

Researcher: So you accomplish this. You’ve got all your colors at all different times, patterns, everything. And somebody comes in and says, "I’m going to measure the quality of your garden. And I ’ve found that the most convenient measure is the length of the leaves. So I’m going to go measure by the length of the leaves and let you know how you do in comparison to other gardens." And then of course, the blossoms don’t count, the colors don’t count, the variation doesn’t count, making it grow on the slope doesn’t count.

And then what happens the next year is that you give the garden a particular kind of fertilizer to make only the leaves grow, and you get no blossoms but you’ve got 290

long leaves. And you have a distorting of what it is you are after in order to look good on this very narrow measure. That’s as far as I’ve gotten. I don’t know if

Administrator: I love that. I love that.

Researcher: But it [education] is so complex. And it gets so convoluted to explain for some...So shall I keep working on that [analogy]?

Administrator: Absolutely.

Researcher: OK. Good.

Administrator: Right to my heart.

In addition to on-the-spot checking of what I thought I was hearing and sharing my own metaphor as a way of revealing my value system, a discourse rule inherent to post-positivist research prompts the researcher to check her written interpretations of what she thinks she is seeing in the culture. As asserted by Heron (1981):

To understand a...culture, we need to participate in it through dialogue and interaction with those who exemplify it. Any cultural explanation needs to be checked with those within the culture, (p. 24)

As mentioned above, 1 summarized the research findings at various points in the study and gave the written summaries to the individuals interviewed to date. The final

February 1990 document described the Foster professional context and the six themes around which I had organized their talk about the state testing policy. In effect it served as a presentation of the collective selves of the

Foster professionals. This presentation, essentially my 291 interpretation of their interpretations of the influence of the policy in their culture, needed to be verified by them to insure its accuracy. In addition to providing for a verification of the findings, this and other methods used in the research influenced the context being studied. The following section describes the perceived effects of conducting the interviews, the written member checks, and the reflection meetings on the local participants.

Effects of Research Methods on Foster Educators

Spontaneous comments during the initial interviews indicated that conversing about the state testing policy with a researcher could be beneficial to practitioners.

Examples of these comments include:

Principal: This is as much for my benefit as it is for yours. In education we don't talk about this enough.

Teacher: It [the interview] is like a brainstorming session.

Assistant Principal: I..talked more than I thought I would. When ..you asked me [for the interview] I thought...'I don't know how much I can talk about testing.' ...I enjoyed talking about it. You know, it’s kind of fun, actually, finally putting out some thoughts that you don’t think you have.

When I sent the initial written member check document in May 1989 I asked participants to respond not only to whether they saw my summary as accurate, but also: 292

- Has my asking about the state testing policy as a part of this research caused you to pay more attention to it and its implications than you might have otherwise?

- What effect, if any, has your reading of this summary had on your view of the state testing policy and its implications for the district?

Generally, respondents noted that the summary was

accurate and interesting to them. They reported that it clarified their thinking rather than changed it, and that

it was useful to see what their colleagues had said about the policy. Talking about the policy in depth provided both

reinforcement of ideas and new insights. One principal was struck more dramatically by the document. He reported that he got so excited reading it he started jumping around and

shouting. In a calmer moment he explained his reaction:

Hrincipal....What 1 need to tell you is that 1 have always had a very strong sense of malaise about testing and how we did it. And 1 never was able to congeal those thoughts enough until 1 read what you wrote. And I said, 'Hell yes, that's what the problem has been'. And it's wonderful that you have been able to focus that, because what you've done is you’ve taken this rather unfocused, uneasy feeling that I ’ve had about it and been able to broaden the focus and you brought up Borne very good points.

In [the summary] you said something to the effect of middle school interdisciplinary teaching, one of the most effective ways of teaching knowledge, is not tested anywhere...and teachers are beginning to resent imposing a top down testing program, not from me but the state, but I ’m just, as you said

Researcher: You are the conduit.

Principal: The agent of the state. 'I’m from the government, i'm here to help you.’... And they’re beginning to resent the fact that not only do they have 293

to teach interdisciplinary units, which they like teaching, but also have to cover all the content, but also have to do testing, but also have to do Project Business, but also have to do sex ed, AIDS ed, health ed, drug ed, you know, "How many things do you want us to do?".

So I was really excited when I read that stuff because what it has allowed me to do.. In fact I ’m going to keep this in my drawer, it's allowed me to focus my concerns. And I guess I never really thought about it.

In fact I was very suspicious of your research to begin with, but - 1 can tell the ’60s in you coming out by what you are doing.- What you are saying is basically a condemnation of the process and this top down mandate. Also what you said was that basically, the California Achievement Test or statewide testing doesn’t test the teaching methodology that we uBe here. Maybe it does for [other cities] or other school districts, 1 don’t know...but the way we focus on the writing process. How can you ask a kid to evaluate a sentence on a test that is incorrectly written, when all w e ’ve done all the way through school with our kids is to have them read their own writing, and other kids’ writings, you know

Researcher: Which is a part of a different context.

Principal: A whole different context. How can you ask a kid...the transference is not there. Yes there is some, and as adults we can see that, but I ’m really concerned about having a 13 year old kid being able to...first of all teachers can't answer their questions, clarification questions, which you also pointed out. Secondly, how can you ask kids to do something that they have not really been exposed to doing before?

So your concern is very valid in saying, what we are going to end up doing is teaching - if these statewide or regional comparisons are made - we are going to start teaching to the test, which is going to take us off our home base of teaching what's best for kids.

Researcher: It's going to take you backwards.

Principal: That's right.

Researcher: That’s what I got. And again, this is a compilation of what people have told me. These aren’t...I'd like to take credit for the whole line of insights but it’s really... 294

Principal: But you put it together. Let me go through theme 1 here...where are your questions? 'In general is it accurate *--yes; 'What additions or deletions, modifications'— I wouldn’t make any. You are right on target here, this is good stuff.

Researcher : What about the question: 'Has my asking about the state testing policy as a part of this research caused you to pay more attention to it and its implications than you might have otherwise?*

Principal: No, it really hasn't impacted my thinking about it. I mean I had no idea where your research was coming from. I had no idea. My impression was that you were going to be here to validate testing as an appropriate thing.

Researcher: Oh.

Principal: I didn’t know that where you were coming from was the opposite direction.

...That's why I asked you about your background. I wanted to find out, where is this person coming from, this is great stuff. It's almost written by a school principal, it’s that good.

The problem is, I can't, I don't have the time.

Researcher: You don’t have the time to do this

Principal: to do this.

Researcher: I can go into my basement with my computer

Principal: And do this, and think about it

Researcher: and pull 15 interviews and any number of observations together into this.

Principal: You see, that's critical, it's so important.

This conversation manifests many aspects of the research process. It points out how the research can serve to clarify the thinking of the practitioners who do not have time to reflect on their beliefs and to bring their 295

collective thoughts together; how the background) beliefs,

and person of the researcher influences what questions are

studied and the manner of the inquiry; and how researchers

are viewed with suspicion until it can be determined "where

they are coming from'’.

When I asked Bill, the assistant superintendent, for

his view of how my asking questions about the state testing

policy might influence Foster educators, he responded:

My sense is the questions anyone would ask about testing who has both the experience you do and the kind of reflectiveness, [are] going to raise issues for people that weren’t there before. That is to say, typically people don’t treat test data with much interest or respect. Frankly I find myself doing that. And when you start asking questions about what its meaning is, what its power is, what it communicates, in some cases you just instigate thinking that wasn't going on before because people weren't paying attention to it, period.

So it has that kind of effect, I think. A person coming into the environment asking questions that weren't asked before, will raise concerns, (pause). I don't know what that means, I only...I feel strongly that you have to have some time, certainly some training and expertise to take this [testing] information and use it in a very meaningful kind of way.

The impact of my research questions on Bill himself was the subject of another conversation. As described in Story

1, he saw my written summary of the issues raised about the proficiency testing at a February 1989 teachers’ meeting as both useful and "exacerbating". Having the research process

remind him that he was "submerging" the educational

question of the appropriateness of the content of the state 296 test beneath the political question of assuring that students scored well was felt as "incapacitating" to him.

Engaging in research of this sort can be a painful process for practitioners.

It can also be painful for the researcher. As Foster moved through the steps of administering and scoring the achievement tests, the central office administrators began to have questions about analysis and interpretation of the results. Our conversations moved to a technical discussion of the properties of different types of scores (raw scores, scaled scores, percentiles, staninesI and the types of computations and interpretations that are possible and not possible with each. These discussions led to a change in the relationship between myself and Bill. The following passages from our jointly written article describe this shift in the relationship, first from Bill's view as the assistant superintendent and 'aubject' of the research and then from my own:

Bill:

As the conversation between Nancy and me developed, her many areas of professional expertise came to light. Our school district was involved in a major reorganization of itB testing program — especially scoring and reporting of student performance on standardized achievement tests — and Nancy's testing expertise was called into service. A subtle but powerful shift in the researcher/subject relationship occurred. I induced Nancy to serve in the capacity of an expert resource person after I learned she possesses technical skills and specialized knowledge which would be invaluable to our preparation of the testing report for the Board of Education. Nancy accepted the invitation to assist us and she worked long hours with the 297 district computer coordinator in preparing test data reports. She also voluntarily took time to enlighten me on various technical dimensions of test data interpretation.

Only at this point did I as the person under constant scrutiny feel I was coming out from under the microscope and thus changing the researcher/subject relationship. At the same time I presumed Nancy agreed to a decidedly participative role within the district because she believed the experience would give her insight into the dynamics of state policy at work on the local level. The distinction between observer and participant blurred; and I questioned if she was actually becoming an activist. I recall we discussed this point without ever resolving it to our satisfaction.

Nancy:

Once the scoring and analysis of the achievement tests were completed, Bill and the district's computer specialist had a number of questions regarding test interpretation. My previous experience in program evaluation and test development was tapped during conversations regarding the technicalities of the different types of test scores, (e.g., the strange things that can happen when raw scores are converted to scale scores and scale scores to percentiles, and how some questions cannot be answered without considering the standard error of measurement for the test). As Bill and his colleagues prepared to present the test results to the school board, they asked for my help in these deliberations. I spent time looking at the results, conferring with them on what the different interpretations might be, and making suggestions regarding the formatting of graphs and the wording of the report to the board. Bill publicly acknowledged my assistance during his formal presentation to the school board in June, 1989.

During this time I felt grateful I was able to "give something back" to these practitioners who had given so much to me. At the same time, I felt a nagging sense of having "gone native", of caring very much that these educators did not get burned by misinterpretations of results from tests which at best assessed a small fraction of the instructional program they provided to students in the district. Some part of my researcher self felt guilty for having these protectionist feelings. After the board meeting, I thanked Bill for his acknowledgment, but also noted that I was embarrassed by his public praise. He seemed surprised at my embarrassment and teasingly noted that "that line between observer and participant gets very fuzzy." 298

Although I theoretically understood that I could not do valid and ethical research within this district without

involving myself in the process, I was still uncertain about the when and how of my involvement. The lack of absolutes and prescriptions for action which characterizes the alternative research paradigm and the vestiges of the old rules regarding researcher detachment were making me uncomfortable during this phase of the study. One way of overtly dealing with this discomfort was to make it a part of our ongoing conversations about the unfolding research process. Such conversations influenced the relationship between us. The changes in the researcher-researched relationship through the various phases of the inquiry is the subject of the next section.

Evolving Researchei— Researched Relationship

In December, 1989 I gave Bill a draft of my summary of

the sixth and final theme, the issues surrounding state and

local accountability. Unlike the summary of the previous

five themes which were written in traditional paragraph style, this document was segmented into two columns, with

lengthy direct quotations on the left and short summary statements on the right. As will be described in Story 3, I wanted Foster educators to see directly their own powerful words about this issue, with as little of my mediation as possible. Upon reading this draft, Bill suggested we work 299

on an article together about our evolving relationship.

Something in my writing made him realize that we had moved

through six discrete phaseB which he identified and named.

From the point of view of the practitioner, the researcher was progressively seen as a:

suspect stranger,

cordial acquaintance

welcome guest

expert resource,

valued colleague, and

confidante.

The conversations between Bill and me generated by the

research process both brought about and revealed these

changes in our relationship. Our first meetings were brief,

and the talk was businesslike and cryptic. As time went on,

the sessions got longer, included more topics, and began to

include revelations of our personal histories, family

lives, and hopes for the future. For example, when I

reviewed transcripts of our reflection meetings, I noted

that in October 1989 we tended to interrupt each other, to bounce back and forth among topics and to have a more

free-wheeling conversation than was typical of our March

1989 sessions. The discourse style of the October meeting

was more representative of a couple of friends working

together on a task; that of March was of business 300 acquaintances uncertain of how the other person would take our next words.

Our research conversations included my raising some negative information, that is, information that 1 was certain would be uncomfortable for Bill. How this information was handled between us became a test for where the relationship was at the moment and how it would develop in the future. An example from our "valued colleague" phase was described as follows in our jointly written article:

Bill:

Nancy gradually worked her way into the fabric of life in Foster Schools. Increasingly, I came to expect her to attend key meetings and to participate in the deliberations. I began to rely on her reporting of meetings as a means to assist me in reflecting on important issues. The more she expressed her interpretations, the more willing I became to share concerns with her and to test my hunches and ideas. At various junctures, I sought her response to strategies I considered using to move the district along in its efforts to comply with the state testing mandate. This advice I viewed as the informed opinion of a valued colleague. The fact that Nancy and I seemed to hold similar perspectives on the problematic nature of legislated learning and its attendant testing mechanisms certainly enticed me to view her advice positively. In any event, our own relationship' thus became dialogical, although I selfishly began to rely on her expert opinion almost as if she were a district employee.

The extent of this valuing was perhaps most obvious when Nancy shared with me her account of my presentation to the Board of Education regarding student performance on the achievement tests. I alluded earlier to her assistance in helping us ready this report for the board. Nancy prepared a transcript of my report to the board; that transcript conveyed to me that I had not made a coherent presentation. I have never reviewed the transcript with her, for it is a source of some embarrassment to me. The transcript revealed that I need to make changes in my future presentations in order to avoid the overuse of jargon and the unintelligible rambling that punctuated some of my reporting. Again, while 301

Nancy and I never discussed this, I learned from her work. In a very direct way, I have learned much throughout the research project.

Nancy:

A "stranger/friend" dilemma of ethnographic research arose with my transcription of Bill's presentation of the test results to the school board. That is, I was a close enough observer (friend) of the culture to know the pertinent question to ask; yet I needed to keep the distance of the stranger in order to have the personal courage to ask the troubling question. I noticed as I was doing the transcribing that the presentation was not as clear, used more jargon and technical terms, and resisted questioning from Board members more than either of us had realized at the time. I wanted to ask Bill about this, especially its implications for the communication processes used in the district (also a focus of the dissertation). Yet I was hesitant to appear critical of him and I did not want to add to the difficulty that these test results and their misuse by the public already made for his administrative life.

My choice of action was to send him the transcript with a note saying that I would like to talk about it. He called back to say that he was willing to talk about it for my research purposes, but had found the transcription both embarrassing and useful. He discovered some of the same difficulties with his presentation as I had and noted that seeing the transcript helped him realize what he would do differently in the future. Although "painful” , he found it to be an opportunity for learning.

Our research conversations had evolved to the point where we felt it was safe to expose feelings of both strength and weakness, confidence and confusion, clarity and indecision. Within this trusting relationship, I was no longer hesitant to surface negative information. I became confident that it would be heard without hurt; that it provided challenge accompanied by support; and it could be dealt with honestly as a part of the personal and 302 organizational context which characterized his work setting and my research site.

Bill now views the six phases of our evolving relationship as a conceptual model with which he can assess the development of other research projects in the district.

He perceives the model as a way to understand, organize and anticipate the different interactions and experiences he has with other researchers.

For me, the six phases tell the story of how I became less of an observer and more of a participant in Foster.

They highlighted turning points of increasing involvement and helped me to understand the changing feelings I was having about the people in the district. I became more concerned with their well-being, and wanted to help them address the professional dilemma they so eloquently described to me as they spoke of the potential harm they saw the state testing policy doing to the education of students. They seemed to need a way to articulate an alternative form of accountability, especially to their local community. As the community became more embroiled in turmoil, the need to articulate an authentic way to assess the quality of its schools seemed both increasingly necessary and unlikely.

As my understanding of their position developed, I began to wonder if the research process might serve them by encouraging their development of an alternative way of 303

being held accountable. As a result, my discourse strategy

became more active. I raised more questions; I started to

ask them how they might articulate their vision of their

schools to the local community and whether they could

develop appropriate assessment indicators that could be

used for local accountability.

My dissertation committee members and I pondered whether the knowledge constructed during the research process might help Foster educators develop an understanding with the local community about the way in which they were educating its children. Heron (1981) described the moral responsibility researchers owe to participants:

Knowledge fuels power: it increases the efficacy of decision-making. Knowledge about persons can fuel power over persons or fuel power shared with persons. And the moral principle of respect for persons is most fully honoured when power iB shared not only in the application of knowledge about persons, but also in the generation of such knowledge. On this view researchers have a moral obligation to initiate subjects into the whole rationale of the research they are doing and to seek the free assent of subjects to this rationale so that, internalizing it as their own, the subjects can become autonomous inquirers alongside the researchers. Put in other words, doing research on persons involves an important educational commitment: to provide conditions under which subjects can enhance their capacity for self-determination in acquiring knowledge about the human condition, (p. 35) (emphasis in original)

I started to hope my research process could help to generate conditions within which Foster participants could 304

enhance their capacity for self-determination regarding the

state testing policy. That is, if they were not satisfied with the way in which the state policy was holding them accountable, how might they develop an alternative form of accountability to be used instead. The use of the research

findings to generate conversations concerning how they might reframe the way in which they are being held accountable is the subject of Story 3.

Summary of Story 2

In summary, the research discourse rules included the

engagement of the researcher in the dialogue, interactions, and work of the participants, resulting in conversations which influenced those being studied and the researcher.

The research conversations and member checks affected

individual practitioners in Foster by helping those

interviewed clarify their thinking about the state testing policy and by helping the assistant superintendent reflect on his own practice. It also influenced the researcher into becoming more of a participant in the assistant superintendent's efforts to first understand the technical aspects of the achievement test data and then to describe the results to the school board.

In the process of conducting the inquiry I came to a deeper understanding of Foster educators' concern for the potential ill effects of this form of accountability and 305 became committed to finding a way for the research process to assist them in framing a new form of accountability.

This process may have resulted in my becoming too much a part of the context 1 was studying and too closely aligned with the assistant superintendent's attempts to minimize what he saw as the negative impact of the testing mandate on the Foster professional culture. As described in Chapter

III, Gitlin et a l . argue against researcher detachment:

If there is a line between "going detached" and "going native", I confess to not having discovered where it is in this inquiry.

Story 3: How might the state testing policy and the research process combine to bring about new conversations?

The findings in Story 3 pertain to my attempt to use the research process to evoke new conversations in Foster about the state testing policy. This inquiry sought not only to understand how Foster educators’ made Bense of the policy, but also asked how, in the realm of possibilities, they might develop an alternative to its approach to accountability. 306

As noted in Story 2, I hold the belief that the researcher has a responsibility to the persons she studies.

In Story 1 the core finding of this inquiry was described as the educators’ professional dilemma over the state testing policy, namely that the use of inappropriate indicators to "measure" the success of schooling could be harmful to the process of educating students. Story 3 asked whether the research process could assist local actors reframe the way in which success was measured, especially within their community. This overall question was addressed in terms of the following specific research questions:

a. How might local educators reframe the way in which the state policy holds them accountable?

b. Can the research process be a catalyst for collective conversations regarding how educators are being held accountable?

c. What organizational conditions facilitate the start of new conversations?

In keeping with the focus on discourse, I had hoped that the research findings would serve as an impetus for collective conversations among educators, especially administrators, about the state testing policy and their response to it. This hope may have been unrealistic given that the demands on central office and building administrators’ time and attention were increasing due to the continuing arrival of new students in the midst of growing strife and criticism from the Bchool board and the 307

community. Within the climate of deteriorating trust and

accusations, there did not seem to be space, time or energy

for administrators to collaborativly discuss the state testing policy. As the schools were being criticized for

their cost to local taxpayers, paradoxically, the administrators did not seem to be able to come together to articulate the criteria with which the quality of their schools should be judged. In other words, the conversations regnrding how they should be held accountable seemed simultaneously becoming more necessary and more unlikely within a context of increasing criticism and strife.

Nevertheless, I attempted to use the research findings to initiate "real talk" in the district, in the hopes that creative solutions for the issue of accountability might be formulated. Belenky, Clinchy, Goldberger, and Tarule (1987) describe how knowledge can be constructed in settings where participants:

...join together to arrive at some new understanding. 'Real talking* requires careful listening; it implies a mutually shared agreement that together you are creating the optimum setting so that half-baked or emergent ideas can grow. 'Real talking* reaches deep into the experience of each participant; it also draws on the analytical abilities of each. Conversation as [knowledge] constructivists describe it, includes discourse and exploration, talking and listening, questions, argument, speculation, and sharing (p. 144).

In an effort to evoke such creative conversations among administrators who seemed to feel that they were alone and 308 unique in their views about the state testing policy, I organized the results of my interviews and observations in a way that would demonstrate their commonalities, and which might prompt mutual action regarding how they were being evaluated and held accountable.

As described earlier, six themes were identified within the first few months of the study. As Foster educators spoke to me and to each other about the state testing policy, their talk focused on: 1) how the tests would be used; 2) the effect of the testing on their professional relationships; 3) whether the achievement tests should be given in the fall or spring; 4) the effect of the testing on their local curriculum; 5) the effect of the testing on local resources; and 6) the issue of local and state accountability purposes. A written summary of all six themes was distributed in February 1990 to each of the 26 people interviewed.

The first five themes have been summarized in Story 1.

In the version sent to the participants, these themes were written in the traditional paragraph format of a research summary. The account of the sixth theme, as evident below, was organized in two columns of text: the left hand column was made up of quotations from Foster educators; the right hand column consisted of a list of their ideas written by me in the form of summary statements. The purpose of this alternative style was to provide for Foster educators a 309

rendering of both their own words and the passion which

they and their colleagues spoke of their vision for what

education should be and the appropriate way in which it

could be assessed.

Theme 6: The Issue of Local and State Accountability

This section includes the findings related to the

accountability issue as they were presented to Foster

educators. J summarised and organized their expressed

concerns around the following six questions:

1. What is the need for accountability? Statewide Locally

2. What approaches are now used to hold educators accountable?

3. What are the problems with current accountability approaches?

Overall problems with testing State level problems with testing mandate Local problems with state testing mandate

4. What would be a more authentic/valid approach to accountability?

Statewide Locally What characteristics would such an approach have?

5. How does statewide testing affect a district that tries to develop a culture of professionalism?

6. What could we do about this issue in Foster?

For questions 3, 5, and 6 the summary statements are accompanied by elaborate quotations in the left column. 310

There is no attempt, however, for an exact left-right correspondence between the two columns. The numbered statements on the right include all of the ideas expressed about this issue by Foster educators. The quotes on the left were selected for their completeness, clarity, coherence and eloquence. Therefore, not every idea included in the summary statement has a corresponding quote, and some quotes from different people express the Bame basic idea. Again, the audience for these findings was the Foster educators. The intent was to demonstrate the commonalities of their views, and the eloquence, passion and creative ideas available within individual administrators and teachers in the district should they chose to harness these toward a collective vision of a new approach to accountabi1ity.

Question 1: What is the need for accountability?

Statewide:

1. The state has the obligation to monitor education in school districts. It cannot just ignore districts, especially those that are not serving students well.

2. Accountability is useful to insure that studentB have skills to enter the workforce.

3. Accountability is useful to provide some continuity of curriculum across Ohio for students who move from one district to another.

4. State policy makers want to use tests to embarrass districts into improving.

5. State policy makers want to know how much it costs to teach to a certain level. Testing is part of a fiscal accounting system. 311

Locally:

1. Taxpayers Bee beautiful school buildings. They need to know the learning that goes on in them, especially learning beyond the basics.

2. The public wants to know that its schools are addressing appropriate issues and student needs.

3. Educators need ways to assess student growth.

4. Parents want to have documentation that their children can compete.

5. Parents want to see how their children are doing over time, for example, 4th to 6th to 8th grades.

Question 2: What approaches are now used to hold educators accountable?

Current measures include:

SAT scores

Number of National Merit Scholars

Competency tests tied to local curriculum

Upcoming proficiency tests for 9th and 12th graders

Standardized norm-referenced achievement tests

Statewide comparison of achievement/ability scores with school buildings and districts identified. 312

Question 3: What are the problems with current accountability approaches?

Overall problems with testing:

"The negative aspect [of 1. How can "success" be defined testing] on the entire in education? And how will educational systee is educators accomplish it? Test that the test scores results do not capture becoie acre important "success" nor lead teachers and than the people ...The students to its accomplishment, people have to be the central focus, not only 2. Tests are only a snapshot of the students and how they a particular day and activity. feel about theaselves and They do not cover the full what their abilities range of a child’s achievement. are..but the staff as Hell.' 3. There are some things that are just not testable, e.g., 'Me need to be able to how a child feels about ■easure and evaluate how himself, helping students students are progressing through adolescence. along the way and [ask ourselves] if the end 4. Students (and all human product is a product that beings) cannot be reduced to a we really Hant. But the number for accountability end product Hill never be purposes. aeasured by a test score. I t ’ll be aeasured by 5. Testing encourages the soeething well beyond sorting of students, even as that." young s b the 7th grade. This is a disservice to students, "For people who don’t especially the late bloomers, have an enotional since they can and do change in investient in their own their approach to education children...[test scores] later on. are only a nuaber...It is only a nuaber to the 6. Achievement tests are well legislators who are Buited for middle class saying 'This is the score students. What is often that [a neighboring reflected in the test scores is district] got; this is the correspondence between Nhat that [another socioeconomic statuB of the district) got. And this children and the tests. is the score that Foster Suburban school districts often got.’ And then what is take credit for students' high going to happen to those scores, when the scores have inner city schools? much to do with students’ socioeconomic status and not the learning that the schools 313

"Children are not have "added" to the children. numbers. He are not a These tests do not reflect the product oriented knowledge added by the schools. business. He are not a business. He are learning centers for people. 7. Achievement tests are People are not numbers measuring learning in an and it’s grossly archaic fashion, ignoring the ■isrepresentational to last 25 years of research on even attempt to reduce how children learn. anyone to a number. So our concern as 8. Students need to develop people..working with self-help strategies to become children is seeing that independent readers. they value reading and Achievement tests do not writing..find it s measure such strategies. impossible to have a number placed on a child to measure that."

"A big part of assessment 9. Non-educators do not realize is kids knowing when they the nature of of the are in trouble and when norm-referenced test and the they are not. And by measurement error inherent in relying on standardized the testing conditions. For tests you are essentially example, tests are designed to taking away that spread out student scores; responsibility from the items that too many children child, where it should answer correctly are not be." included in the final version of the test. Standardized "It's a demoralizing, testing conditions cannot negative experience for prevent electricity going out; the learners because they kids recovering from chicken are programmed to lose. pox; students finishing 3 weeks The average person is of working on an inter - going to get 50* disciplinary unit before they correct." are tested on isolated skills in discrete disciplines; testing students on non-calculator math when they have just finished working with calculators. Teachers cannot insure that students will even read the question before choosing an answer, or that low achievers will try to do their best on the test. State level problems with testing mandate:

find when the state cones 1. State and national tests do in and says, ‘You nil) not match local curriculum. lake up your own test and There is no one answer to what you can test your own should be taught and when, but curncului but the bottoi standardized tests assume there line is we are going to is. Statewide testing pushes us rank you with other to a Btatewide curriculum. districts in the state', they are working toward a 2. To give a standardized test statewide curriculun." when you do not have standardized curriculum, ‘[The state testing standardized textbooks, or reflects) a factory standardized lessons plans, and •entality, that everyone you do not have the same has the sane raw student needs or the same ■aterial...everyone community values, does not make starts in the sane place, sense. It is not possible to everyone ends at the saie validly simplify information place, with the sane from diverse settings into a product. And that's not statewide aggregate, when Nhat we have. So the different schools are tests don't aatch the responding to different children and the children community and student needs. don't natch the test and so ...we spend tine 3. Why are we doing this? Using teaching to the test or educational dollars to compare we let the test define students on test scores does us...What then becones not help students learn more, the rule of thusb is not even in those districts where w h a t ’s best for the test scores are low. State children, but what's best testing is for political to nake us 'look good' in purposes, not instructional the factory line." purposes, and only the testing companies benefit. Emphasis "In the snaller districts should be placed on preparing and in the urban local Btudents, not comparing districts where the kids students across the state and are going to do aiserably the nation. [on the tests]...it is going to depress the 4. There is a state and schools even further.* national assumption that increases in test scores mean an improvement in schools. This is not necessarily the case. 315

5. Non-educators (especially "The whole thing is so businessmen and lawyers) in the ridiculous..when you have state legislature and on local legislators who Know boards are making decisions nothing about how about what needs to be done in children learn. People education, despite the fact just don’t trust teachers that they know nothing about enough; they definitely child development, how children don't trust learn, curriculum, instruction administrators. or non-linear approaches to developing children’s thinking. "I think these people are looking [at education} as 6. Schools which go beyond the a business, (looking for] lock-step thinking reflected on a product, and we don’t tests can get punished for not have a product. He don’t doing well on the tests. Yet see a visible product this creative thinking and when they leave here..He risk-taking is just what is hope that they have needed to respond to society’s learned their skills and current and future needs. we get surveys back (telling us)., that they 7. The state insertion of the are doing Ok, but we mandated achievement tests don’t see that product. midstream in the process of developing competency based *1 view the legislature assessments kills the positive ...as a bunch of effects of the competency tests businessmen, by and and distorts the information large, who use industry's coming from the achievement fomat of productivity tests. It changes the purpose based on competition. And of the testing. Instead of the way we are going to assessing what you are measure productivity is teaching, you will now have to some statistical means insure that you "look good" on [like] statewide the achievement tests. testing.., And it really shows a lack of foresight 8. The Btate testing policy on the legislature’s part could lead to law suits if of what education is all students claim that schools did about in this country.' not educate them properly. If "properly educated" is defined 'Lawyers are not on the basis of the proficiency educators. It really tests, rather than be left angers me, the fact that undefined and nebulous as it attorneys, lawyers, state was before, schools are open to representatives think being sued if students do not that they can write pass the test. educational policy.' 316

Local problems with state testing mandate:

"[Tests] Beasure tilings 1. Standardized achievement that are easy to aeasure, teats do not include all that [they] don't aeasure Foster teaches. The local learning. curriculum goes far beyond the basics assessed on the test. "Our whole iapetus with Foster Btudents are asked to kids, froa the tiae they generate answers to complex walk in this door, is to problems, to generate writings, aake thea independent, not just to select an answer or self-reliant individuals, to edit someone else’s writing. Mho want to do things, not n o o are aade to do things." 2. In addition to including "What |the achieveaent more than the test measures, test] tests is not uhat I the Foster instructional think is liportant for ay program often teaches the classrooa." content on the test differently than the way it is measured on "He are looking at auch the test. It is unfair to test aore sophisticated skills students on approaches they when tie assess learning." have not been taught.

"He've got to quit using 3. Our purpose is to make tests that teach lower students more self-reliant level thinking to set our regarding learning, to make curriculua." them independent so they want to do learning activities; not ‘I'd thinking of soae of to make them do the activities. the s t o n e s [on the State testing works against test]. He thought we had this approach. the wrong foras for the third grade for a while 4. Given that the state and I’a not too sure that mandated test does not measure that didn't interfere the content or the form of with the tests scores... local instruction, and that the [The test] was like: test resultB will be compared 'John and Hary were across districts, schools, and playing ball. Bill was perhaps eventually, classrooms, riding his bike’. The teachers may feel compelled to question would be: 'Hho teach to the test, despite was riding the bike?' And their professional judgement the kids were saying, telling them that this is not 'Hell, where’s the what is best for students. 317 trick?1 ...After a day of this the teachers were 5. Teachers will feel pressure asking if ...we have the to teach to the test in order wrong foras... These kids to “look good” and to respond are being blown away to the community which will because they c a n ’t want documentation that their believe that their answer students can compete. can be right because it's too staple." 6. It is frightening to think "He have very few that parents could look at the workbooks in this school. test scores and conclude that So kids need to learn their child is not as talented test taking skills." as they thought he/she was, without ever knowing that there 11 he tests could be used is a mismatch between what is to sayj this a better taught and what is tested. teacher than that one..Teachers still have that fear, which is why they teach to the test.

"If these statewide or 7. Parents who called were regional coaparisons are confused about the difference aade, we are going to between percent and percentile. start teaching to the They thought the percentile test, which is going to reported on the forms sent home take us off our hoae base reflected the percent of of teaching what's best answers their child answered for kids." correctly.

"You see the goal changes 8. What could this year's then. The goal becones to achievement test scores show up good test scores possibly mean to the school on that test. It d o e s n ’t board? What would the scores be becone an educational compared to? It would not make goal. It becoaes a aatter sense to compare them to laBt of survival. He are year'b scores since there have throwing education out been so many new students, new the door and we are schools and new attendance talking about how is this areas since then. school systeB going to survive. This is what the state's expectation is of ae, and I’ve got to aeet that expectation. So I will do whatever is necessary to aeet that." 318

“llestingj is not Mhat [education) is about and i t ’s not Mhat 1'a about [as a teacher). But in the back of ly aind I’a going, 'These kids are going to boab [on the test] and then...people are going to use these scores to aake judgeients about [the students]"',

"In the future if they record these scores according to districts, according to schools..1 think mb have a reputation for excellence...we are challenging each kid... I Mould not want test scores to show otherwise. 1 Mould try to bring those test scores into line so that they reflect Mhat Me feel on a day to day basis is good things happening for kids."

Question 4: What would be a more authentic/valid approach to accountability?

Statewide:

1. Foster educators suggested that each district might set a standard for itself which addresses the needs of its students and the state's role would be to monitor whether each district met its own standard.

2. The state could continue the approach of having local districts develop competency based assessments aligned with their courses of study (and drop the Btatewide tests which imply that all districts are alike and can be compared).

3. The state could give some districts more attention and others less depending on their paBt performance, just as principals give more supervisory attention to some teachers than others. The difficulty here is that a 319

district may first have to perform well on a measure it views as an invalid assessment of its instructional program in order to get "off the hook" of further scrutiny from the state.

4. How the question of accountability is framed by the state is important. If it is framed as a top-down authoritative mandate, it could breed rebellion on the part of school districts. If it is done in an atmosphere of mutual respect and professionalism, it could evoke cooperation.

Locally:

1. Given holistic teaching and learning, how do you do multiple assessments of the process? And how do you aggregate, summarize and communicate the information that comes from the multiple assessments?

2. Since standardized tests (achievement and proficiency), SAT scores, and the number of National Merit finalists do not measure the Foster curriculum, it is important to tell parents how students are doing in the curriculum beyond what these indicators measure.

3. Ultimately, accountability rests with the teachers, but teachers need to be seen as professionals for that to occur.

4. What would make sense for local accountability is for teachers to develop the curriculum that serves local students, and then develop assessments of this curriculum. No one measure would be sufficient; multiple measures would be needed.

5. Competency based assessment related to local courses of study is a fair and legitimate form of accountability.

6. A valid assessment of reading would be to focus on the true purpose of reading, namely, comprehension. Use passages from real literature to assess comprehension. A writing Bample would be a valid assessment of language arts, with mechanics checked separately from expression. Mathematics assessment would focus primarily on reasoning with some computation items.

7. It’s difficult to think of a cost effective way to conduct valid assessment. Writing samples, for example, are expensive to evaluate. 320

8, The best communicators for local accountability are students who talk at home about what they are doing in school. If parents see that their children are excited about what they are learning and feel good about themselves as learners, parents will not be too concerned about test scores. School board members may still be interested in test scores, however.

What would be the characteristics of authentic/valid accountability:

1. The issue would be refrained from 'accountability to 'responsibility* for the student’s learning.

2. Accountability measures would be noncompetitive.

3. There would be multiple measures related to the local curriculum.

4. There would be an emphasis on direct assessments (those interwoven with the daily instructional process) rather than indirect or contrived measures away from the teaching/learning process. Examples of direct assessments related to reading instruction include:

-teacher anecdotal records about the student's progress; -comments from others who work with the student, including parents; -notation of how child retells a story he/she has read; -notation of behaviors student exhibits while reading; -checklist completed by an observer while student reads *, -products produced by student, including journals, audiotapes, selected pages from written work, videotapes of student reading; -student self-assessments; -portfolios of students* beBt work over 12 years.

5. Elevate the importance of narrative descriptions of child's progress and teacher’s anecdotal records. For this to occur there must be trust in the teacher*b observations of student learning.

6. Incorporate students’ daily work and teacher's criterion referenced classroom tests as part of district assessment procedures.

7. Include students in assessments of their own work. 321

Question 5: How does Btatewide testing affeot a district that tries to develop a culture of professionalism?

'It's aiazing though, the 1. Over 120 Foster teachers are public expects, the involved voluntarily in a legislature expects, the course on improving reading board of education instruction and some of them expects, principals are pushing to use the ideas expect, everyone expects generated in the university teachers to be top-notch course to revise the local and we want all these course of study in other indicators of their curriculum areas. They are performance..[such as) eager to make curricular and testing..but He don't instructional changes because give them the wherewithal of their commitment to the to become the very best students in their classrooms that they can be. Because today and every day. They ne lock thB# in this room cannot wait until the *time is all day and they can’t right' or it is politically and get out of it. administratively convenient. In the teachers’ view, the They can’t research, they children need the best can’t reflect, they can’t education possible now. go to the places they need to go, and can't 2. The way to improve observe other people, instruction is not by testing, they can’t do any of this but by giving teachers the because they have to be opportunity to grow, and to in this classroom all invest in them and pay them as day..He have to give then professionals. the wherewithal to grow. And that’s Khat’s going 3. The local administrative to inprove teaching, not goal is to develop leadership test scores. Teachers in all teachers, to strengthen need to get out of this their diversity and encourage isolated cubicle that each teacher and each student they are in and start to be the best possible in the groHing professionally. areas of their strengths. In So the answer that the contrast, the state testing legislature should be policy pushes everyone to preoccupied with is not sameness, toward mediocrity. test scores, but giving Such a policy is looking for teachBrs the opportunity the median, an average, not to to do this," different people excelling in 322

different, ways. This policy is "[This business!ike] thus a contradiction to what wanting to put a nuaber education should be about. on the thing and aeasure that, and sake you 4. The average person thinks coapete for the that "anyone can be a teacher". nuabers..this brings out the worst in people, 5. The state testing policy rather than the best in assumes a factory mentality. It people...The potential assumes that educators start good that could coie out with the same "raw materials" of statewide testing is and turn these materials into a coapletely overshadowed "product" while using by the negative aspects competition in the process. of doing ooapetitive statewide testing...! don't see competition as 6. Competition in education healthy within the brings out the worst in people. educational systea.' Rather than people who are competitive, the work place and Bchools need people who know how to be cooperative. ‘I don’t see [education] Competition fosters a climate getting any better of blame from teacher to because of competition. I teacher and administrators to see us using teachers. It creates a [competition] against situation where pressure and each other. 1 see it in blame are passed from the levy propaganda..it’s school board to the one-up-«an-ship. find 1 superintendent to central don't think it's going to office administrators to work, but you are going principals to teachers to to see it happen [to students. schools]. 7. Comparisons based on test 'Let's say for example scores encourage teachers to that the board of throw out the local curriculum education becomes and teach to the standardized concerned because of low test. scores. They they put the pressure on the 8. The long term effects of superintendent, who will such comparisons and then put the pressure on competition are negative central office because they focus educators* administrators, who will attention on test scores rather then in turn put the than on people. pressure on building administrators who will 9. The state testing policy pass that along to the assumes a military mentality teachers. And then the among educators. It assumes teachers in effect will that administrators should tell say, 'kids, we need to teachers "Just do it", whether 323 buckle down and get back they agree with the testing or to the basics.' It's like not. passing the buck. Hopefully, that scenario 10. The state testing policy is Non't happen in Foster. far removed from, and does not But it could very Kell take into account, the life and happen." death issues that educators must deal with every day in "There is enough of this their work with young people. [blating] in education already. Middle school 11. The role of the principal people are saying, 'Ah, is to minimize the impact of the eleientary people the state testing policy so d i d n ’t do their j o b ' ; the that teachers can still do what high school teachers are is best for students. saying, 'Hell the eiddle school teachers didn t do 12. There is a lack of trust in their j o b . ’ There is teachers which grew out of the enough of this already. I "reform" movement. People d o n ’t think k b need to legitimately want competent foster anyoore. And teachers and effective leaders. (testing] certainly Kill But using testing as a means to do that." reach those goals is dysfunctional. Instead what is "If he try to continue to needed is professional run schools as businesses development for teachers and run business we are going administrators. to be in trouble because kids are reduced to being products rather than being people. And that's very pervasive in how we approach testing in schools."

" I ’d like to say, 'Hell, He a r e n ’t going to have a state nandated testing prograe drive the curriculu#’. I think everyone likes to say that. HoHever,.. I think teachers here in Foster are knowledgeable about the ratifications of testing once the test results are aade public. They are going to be held accountable."

"hy philosophy on education is that it's the people nho have to be the central focus..And the statewide 324 testing...breaks down those priorities; it focuses on tests rather than on people.

‘On the day you called [to talk about state testing policy] 1 nasn't thinking about state Hide testing. I Has thinking about the boy Hho got killed in a car accident...And that's what these [legislators] don’t knoH anything about. Me’ve wrestled with the boy’s death all Neek and hb’11 wrestle with it as we put hie to rest today...But the legislators don't know. And it has nothing to do with state Hide testing. People on the outside d o n ’t know what we go through; what kids go through."

Question 6: What can we do about this in Foster?

Political actions:

'(The) scary thing is He 1. State testing policy has continue to do these educators doing things things because they are politically that are not in the politically savvy, not best interest of students because they are educationally. It is important educationally sound." to have these two different understandings of the benefits ‘1 Hould like to be the and the harm of standardized first district in the testing collide in the public state to stand up to the arena in order to articulate state and say 'Me are not this problem and bring about a testing, p e r i o d ’.* change.

‘And in a nay this kind 2. Educators have not of thing is healthy too, articulated what it is they do because it’s a nay to and what are the valid ways to educate the public about hold them accountable. testing. Froi every horrible thing, a good 3. The essential problem is how sosetiaes coae out of to communicate about the it." complexities of learning across 325

'With state Hide testing thousands of students in widely he in a sense sold our different situations. Educators souls to the legislature want to deny standardized to let thei do Hhatever testing as a valid way to be they want to Hith. held accountable, without Because if Me can have denying the need for CBE (coipetency based accountability. education] not even fully iapleiented and coee out 4. What could be done at a uith state Hide testing, statewide level is to have each the legislature can do district develop a standard for anything they uanl to itself on which it will be Hith us. And 1 think that judged. The state role could be one of the reasons that to see how the district met its education throughout the standard. This approach would country is in Mhat kind take into account the school’s of shape it is in, is need to respond to the because mb have been too different conditions and soft Hith people, and He communities that students live have been very in and would also preserve non-assuaing, and sit local control. It also provides back and let people do for accountability without Hhatever they Hant to unfair comparisons across Kith us. districts.

And ue are not good at telling people, "NO" ne're not going to do that and these are the reasons that He are not going to do that. And take us to the Hall, but don't expect us to do something that He knovt is not good for kids."

"He are our onn norst eneiy. He don't do a good job of articulating exactly nhat it is that He do and hon He do it to the public."

‘Hhat He need to do is to get auay froa only being able to articulate the ■ost rudiaentary part of our task at hand here, and take people to those other levels. That’s the challenge of bBing an 326 educator in the 2000s. He have to get to the point of being able to articulate those other areas....He are going to have to be able to articulate the subjective data that xe currently have that gives us a reason for being xhat ue are today as an educational system...And xe are still caught up Hith a legislature that Hants numbers to shox [what ne do]...And it is that kind of philosophical barrier betueen us and the legislature that currently has never been scathed,'

Assessment actions:

'[He need to] identify 1. A first step could be to viable aeasures of compare our local instructional ..assesseent for reading objectives with the objectives and composition as a included in the standardized [first] step to tests and see where we match eventually help the and where we do not, and then public understand that ask ourselves why there is a there are lots of difference. This process would elenents to a school serve as a check and balance success profile; it's not against national, rather than just SAT scores, National just local, expectations. Merit Scholars, or California Achieveient 2. Giving everyone the same Test scores. group tests reflects the previous factory mentality, 'The problea is there which assumes that every isn't a better student starts at the Bame replaceient right noH... place and ends at the same He need soaething [for place with the same 'product'. accountability], so He In the information age we need use this [standardized assessments that are testing].' sophisticated enough to look at 327

students when they start, at 'Real assessment is not a guideposts along the way, and yearly test. Real when they reach their assessaent is a daily destination. Such assessments occurrence in the would be focused on the classrooB...'1've taught individual. this lesson, have the children learned it? If 3. It takes time to develop and not, Hhat can I do? Hhat refine authentic assessments of about Josie? Kell, he what Btudents are learning and knows it. Hhat about to educate adults in and out of Eddie who probably Hon’t schools about these valid ways know it for three aonths? to measure growth among It's ridiculous to send students. Eddie and Josie in to take the saae test. It doesn't serve their needs at all.'

"It takes tiae to develop all the instruients of the assessaent...but what else are we here for?"

Research and Collective Conversations

The quotes provided above demonstrate the clarity and passion with which Foster educators talked individually to the researcher about the policy. The question for Story 3 is whether the research process could serve as an impetus for educators, especially administrators, as a group to talk to each other in a new way about the policy and their collective response to it. There were several possibilities for such conversations over the course of the study.

As noted above, during fall 1989 the assistant superintendent and I began discussing with a local university the possibility of offering a course which would 328 bring teachers and administrators together over several university quarters to review various forms of assessment

{e.g., classroom assessment, standardized testing, alternative performance measures) and to discuss how these could be used to articulate an "expanded accountability statement" for Foster. I was to share in the teaching responsibilities for the course which would have as one of its outcomes the formulation of a set of indicators which

Foster educators would view as appropriate methods for assessing the quality of their schools. The course would simultaneously address the questions of what is authentic assessment (for instructional purposes) and how can such assessment be appropriately used for accountability purposes. In short, what forms of (instructional) assessment made sense for (political) accountability?

Although the idea of providing such a course has not died altogether, it is unlikely to happen given the slow response of the university and the nature of unfolding events in Foster. These events will be described shortly.

Another possible occasion for group conversation about the state testing policy occurred in November 1989 when 3 offered to share the summary and quotes about accountability reported above during one portion of a regularly scheduled meeting of secondary administrators.

When the assistant superintendent checked on the possibility of putting this topic on the meeting agenda, a 329 number of administrators responded that, although they

liked the research 1 was doing, they could not "talk about testing now". That is, they were feeling so overwhelmed with developments in the district that talking about testing policy was not a priority.

The ongoing developments in district level politics of

Foster included the election of two new members to the school board in November, 1989. Not only has their arrival not healed the wounds of past controversies, but their focus has been almost entirely on the financial standing of the district. Board members have told administrators not to talk to them [board membersJ about instructional programs but only to talk about costs. That is, the deliberations on such matters as cutting staff or not building additional schools could no longer include the potential consequences of these decisions on the daily instruction of students, but only on what the potential cost savings will be. Thus the conversations among administrators and board members have been further constrained by a new rule for the discourse: focus on the cost rather than on the quality of the instructional program.

As a result, the context for Foster administrators and teachers has been filled with unease, uncertainty and rumors about which positions would be cut. The atmosphere has been described as "grim" and administrators have been

"careening" from the implications of these events for 330

people, programs and the professional culture they have

tried to create in the district.

In order to maintain current programs and accommodate

the newly arriving students, the May 1990 permanent

operating levy request would have had to be over 4 mills higher than that supported by the school board at its

February 1990 meeting. As a consequence, during late

February numerous teachers, administrators, secretaries and custodians received "suspension of contract" notices, essentially telling them that their services would not be necessary for the 1990-91 school year. The assistant superintendent described his role as "the angel of death" as he and principals distributed these notices.

Given this local context, what role can a researcher play in bringing about new conversations to create new forms of local accountability? As noted earlier, the attempts to start a university course or to include such a conversation as part of a regularly scheduled secondary administrators meeting did not work. Another attempt was made when 1 distributed the member check document of the six themes in late February 1990 (just after the suspension of contract letters had gone out). In the letter accompanying the 45-page single spaced document, I invited the 26 people interviewed to look especially at the accountability section and to give me feedback on the whole or a part of the document in one of three ways: 331

- Call me at [phone number]; or

- Write comments on the document and send it in care of my name to the Central Office where I will pick it up; or

- I'll be in the professional library of the Central Office on the following dates and times. Please stop by for as long or short as you like for conversation with me and your colleagues about the ideas in this summary:

Wednesday, March 7, 1990 3:15 - 5:00 Thursday, March 8, 1990 3:15 - 5:00

Three people responded by calling me; another one sent the document back with comments via central office mail; and one principal conveyed a message through the assistant superintendent that after reading the summary of the six themes, "he will never test again", a promise he is, of course, not at liberty to keep. Only one person came to the sessions offered in the central office library, and although the two of us had a discussion of the various ideas in the document, there was no collective discourse among members of the Foster educational community.

There are a number of plausible explanations for why the collective discourse has not taken place. One reason is the fact that this research, although focused on the sense-making of Foster educators, has not been of them, or by them, and only tangentially, for them. Although I tried to make this effort useful to members of the Foster community, it remained my dissertation, initiated and carried out primarily by me, with some collaboration by the 332 assistant superintendent. The research process has always been in Heron’s (1981) terms, a "weak" form of collaboration, "in the sense that the subject is thoroughly informed of the research propositions at all stages and invited to assent or dissent" (pp. 19-20) (emphasis in original). Given that they did not initiate or conduct the research, an invitation to discuss its findings may not have been very compelling.

An additional consideration is that the document given to them for feedback was quite long (45 single Bpaced pages) and the emphasis of my request was for them to tell me "if I got it right". Given that the ongoing feedback I have received throughout the study had always affirmed my written interpretation of their spoken words, most readers may have not found anything "wrong" which they wanted to correct in the final document. In other words, if there were no glaring errors they may have felt no urgent need to get back to me.

An even more likely explanation is that given the length of the document, and the enormous demands on educators’ time in the best of circumstances, teachers and administrators may not have had the time, energy or inclination to read the document in the current tumultuous atmosphere in Foster.

Furthermore, my cover letter invited them to a district-wide meeting situated in the central office. This 333

may have been a strategic error given my earlier

description that the district was characterized by havens

of professional support within schools but not across

schools. Although committee meetings to revise courses of

study were occasions to work with educators from other

buildings, in general, teachers and principals attach their

professional identity to their own school rather than to

the district as a whole. District-wide discourse is thus

inhibited by a sense of identifying with and protecting

one’s own building. This situation has been exacerbated by

both the phenomenal growth of the district and the feeling

of being attacked by the board and community. When feeling

criticized it may be easier to retreat even further into a

haven which supports and verifies the worth of one’s work

rather than risk discussing a new way to be held

accountable for the district as a whole.

In addition, it may be that administrators at this point in time under these conditions cannot envision what

an alternative approach to accountability might be. The

task may seem just too overwhelming and just too nebulous

to undertake. Moreover, the top leadership in Foster has not identified the development of an alternative approach to accountability as among the district priorities. As the assistant superintendent notes: "Someone has got to authorize it [the state testing policy] as a topic that requires [administrators'] time and attention." 334

The need for leaders to clearly and repeatedly

articulate the vision, mission, and goals of an

organization in direct and symbolic ways has been the

subject of much popular and academic literature, (See, for

example, Bennis & Nanus, 1985; Sergiovanni & Corbally,

1986; and Smircich & Morgan, 1982). The district's top

leadership exercises a key role in the manifestation of the

discourse rule pertaining to what gets on the district-wide

agenda. If the superintendent does not focus on the

articulation of an alternative approach to accountability

as an important task, then central office and building

administrators may be hard pressed to accomplish it alone.

Instead, the other legitimate topics which are on the

superintendent's agenda receive their time and attention.

As of this writing, one aspect of the research findings

may be used by a central office administrator to spur a

discussion among principals. The director of elementary

education would like to use the description of the

individual havens within buildings but lack of collective

support across buildings as a springboard for conversation

during a summer retreat. However, the more general ideas about the Btate testing policy generated by these Foster

educators and made public by the research process are not, at present, a topic for conversation among themselves. 335

Important Conditions for Collaborative Discourse

A further analysis of the combination of state policy,

research, and conversation in Foster may provide a deeper

reason for the lack of collective discourse about the state

testing policy. If the Foster school district is made up of

havens of support within schools but not across schools,

the Btate testing policy spotlightB a vulnerable aspect of

the context: the lack of a district-wide consensus on how

they should be held accountable. In other words, the impact

of the state testing policy may be more keenly felt because

it touches on the weak and nebulous link across schools. It

threatens the well-being of the district as a whole because

it asks the question of accountability about the aggregate,

that is, about the students and schools as a whole.

The state testing policy does not ask the question of

accountability close to the daily instruction in

classrooms. If it did, individual teachers could probably

articulate accountability in terms of individual students.

That is, teachers could hold a dialogue with parents or

others which conveys what an individual child is learning using all of the assessment tools available at the classroom level (including classroom tests, the child's papers, observations of child's verbal expressions, non-written work, etc.).

But the state testing policy asks the question of accountability about the aggregate. The indicator of 336

"success" is removed from the assessment of individuals and focused on the collective. In other words, the state testing policy shifts the focus from the instructional issue of assessment to the political issue of accountability. Principals and teachers may be able to answer the assessment issue about individual students using the assessment tools directly tied to classroom instruction. But Foster educators have a harder time with the aggregate issue of accountability because the indicators of success required by the state policy

(achievement and proficiency test scores) do not match the instruction they have offered. Furthermore, these indicators assume a form of schooling which is contrary to

Foster’s professional vision of how to educate children.

In addition, their professional discourse is more developed within each building than across buildings. To make matters worse, the state testing policy promises to compare schools with one another based on the test results.

The ensuing competition places the well-being of one school against the other and reinforces the tendency to identify with and protect one’s own building.

Table 5 lists the dimensions of this analysis. On the one side, the individual schools with their focus on assessment of individual students for instructional purposes are characterized by strong, healthy relationships among the educators within the buildings and a well 337 developed professional discourse. On the other side, the political nature of accountability addresses the aggregate aspect of the district, which is characterized by weaker relationships among administrators, strife among the school board and community, and a less developed professional discourse across district administrators and with board members. The state testing policy impacts this context via the weaker, aggregate side, effectively challenging the professional culture of the individual schools through the lack of a collective district-wide rebuttal of the policy as an appropriate measure of the educators' efforts.

Table 5: State Testing Policy and the Individual and Collective Aspects of Foster

Individual District as a Schools: Whole:

Assessment Accountability (Instructional) (Political)

Strong Weak relationships State testing relationships across schools and policy focuses within schools strife among board on district as members and a whole and between community compares and board schools to each other

Well developed Underdeveloped professional professional discourse among discourse across teachers and administrators and principal with board members 338

Given that there is not a collective haven among the

principals, central office administrators, and board

members, they have not held the collaborative discourse

needed to articulate a more appropriate measure of success

for their schools. In effect, the form of the discourse may

be somewhat dependent on the nature of the relationships in

the organization. With regard to Foster, I am suggesting

that without the establishment of trusting relationships,

it is difficult for these administrators to have the

conversations needed to construct a collectively defined

alternative to the state testing policy. As a result they

are currently left without indicators of accountability which both they and the local community recognize as

legitimate measures of the quality of the Foster schools.

Within a trusting relationship, within a professional

haven, people can engage in what Edelsky (1981) describes

as a "collaborative discourse", that is a discourse in which several people are "on the same wave

lengthsharing in the creation of an idea" (pp.

393-394). As described at the outset of this story, Belenky

et al. (1986) label this type of conversation "real talk"

in which knowledge can be constructed by participants as

they carefully listen to each other in an environment where new ideas are allowed to grow.

Settings for "real talk" may be well established within the professional culture of the individual Foster schools 339

as teachers and administrators engage in conversations

about teaching and learning. The next step for Foster maybe

the development of trusting relationships across schools as

an important precondition for the collaborative discourse

required to formulate more valid approaches to

accountability. If they could engage in district-wide "real

talk", Foster administrators might jointly construct the

knowledge they need to articulate a response to the state

testing policy. With a valid alternative on hand, they

might be able to resist the indicators of accountability

promoted by the state testing policy without denying the

need to be accountable to their local community and to the

state of Ohio.

Summary of Story 3

To summarize, individual Foster educators expressed

articulate and passionate ideas about the difficulty of

using the state defined tests to hold them accountable for

local curricular and instructional practices. Many of them had ideas regarding alternative possibilities. The

collective conversations needed to bring these ideas

together to construct a district-wide alternative response

to the state testing policy have not occurred. In the midst of a rapidly growing school district, characterized by

increasing criticism and strife and the laying off of

school district personnel, the researcher*s attempts to 340 prompt such conversations were unsuccessful.

Among the reasons for the lack of collective conversations about testing policy at a time when local educators sorely need to articulate their success in educating students may be the lack of trusting relationships across district administrators and school board members. To engage in collaborative knowledge construction, participants need to feel they will be listened to, that their emerging ideas have value, and that together they can create something worthwhile. Without trusting relationships to provide the aetting for these interactions, ’’real talk" about their vision of education and an appropriate means to assess it may not be possible.

Summary of Chapter IV

The findings of thiB inquiry have been presented in the form of three stories. The first story told of the individual and collective sense-making of Foster educators regarding the state testing policy, including their appreciation for the legitimacy of accountability demands on educators coupled with their reservations over the state dictated tests as appropriate measures for the curriculum and instruction taking place in their schools. The policy created for them a professional dilemma in which they had to choose between doing what in their Judgement was best 341

for the education of students and doing what would make

them and their schools "look good" on the tests. The policy

was already affecting their instructional practices* may

affect their written curriculum, and, if comparisons among

schools or teachers are made on the basis of test scores,

the policy has the potential for hurting relationships

among educators, between teachers and students, and between

educators and the local community. In addition, responding

to state mandated testing requirements consumed human and

financial resources which then were unavailable for

developing alternative assessment and accountability

measures.

Conversations between educators and constituents

regarding testing were constrained by the conflicting rules

for discourse inherent in the policy and in the

professional culture. Given that the policy demanded

compliance from professionals accustomed to participating

in educational decisions, and it encouraged comparisons across schools on the basis of test scores which educators

saw as a misuse of test information, administrators were not able to clearly articulate their vision of schooling in

Foster and the mismatch between the assessment instruments and their vision.

Story 2 told of the influence of the research process

on the discourse about testing in Foster. The research conversations helped individuals clarify their thinking about the policy and helped the assistant superintendent reflect on his practice. These conversations also drew the researcher into the work of the participants. I helped to explain and interpret the achievement test results and helped the assistant superintendent prepare his presentation of the test results to the local school board.

In addition, I became committed to helping Foster educators articulate an alternative means of being held accountable.

Story 3 presented the expressions of Foster educators regarding the issue of accountability. It described my attempt to initiate collective conversations to start developing a local alternative to the state testing policy.

Given the current conditions in Foster, including the lack of district-wide supportive relationships, it does not seem possible to generate such conversations at this time. 343

CHAPTER IV

NOTE

1. The terms 'course of study' and 'curriculum' are used interchangeably to refer to the Foster documents which outline the overall philosophy, program goals, and program and subject objectives which guide the teachers' daily lesson planning. The term 'curriculum' as used by Foster educators when talking of the state testing policy does not directly focus on the hidden or silent curriculum, but rather on the overt, intended learning outcomes as described in the district's course of study documents. CHAPTER V

CONCLUSION

Summary of the Study

This case study examined the interaction of language, organization, and policy through a focus on the discourse of educators in a school district about a new state policy.

More specifically, the research sought out the individual and collective sense-making of administrators and teachers as they enacted a state testing policy within the professional culture of their district organization. The perspectives of the educators toward the policy were revealed partially through their discourse. Thus the study focused specifically on the conversations surrounding the implementation of the state testing policy.

In addition to pursuing how a recent state policy influenced the work and relationships of local educators, the inquiry examined how the research process itself influenced the local conversations and explored whether research might serve to initiate new conversations within the organization.

The inquiry was philosophically and theoretically based on the presupposition that social reality is socially

344 345

constructed through the symbolic interactions of human

actors. In this process language and culture influence the

way in which particular human actors construct their

particular organizational reality (Bateson, 1972, 1979;

Berger & Luckmann, 1966; Blumer, 1969; Hall, 1976; Sapir,

cited in Mandelbaum, 1951; Whorf, cited in Carroll, 1956;

and Wilden, 1987).

The study was methodologically based in the

interpretive paradigm which is interested in understanding

the meanings people give to the phenomena that make up

their world. The research methods focused on uncovering how

local educators made sense of the newly mandated state

policy and how they incorporated it into their personal and

collective belief systems and into the professional culture

of their school district. By spending the year prior to the policy’s effective implementation date attending district meetings, interviewing administrators and teachers, holding

reflection sessions with a key informant, reviewing documents and drafting summaries of the findings which were

then checked for accuracy by the participants, I developed an understanding of how these particular educators interpreted and initially enacted the state testing policy.

The inquiry also incorporated some of the tenets of the critical research paradigm. By involving myself in dialogue and interactions with the participants, and by joining them 346

in some aspects of their work, I engaged the study's key

informant in a critical reflection on why he came to

interpret the state testing phenomenon as he did. In addition, I became increasingly committed to helping local educators articulate their vision of schooling and develop alternative accountability approaches which they might view as legitimate and appropriate for assessing their

instructional practices.

With its focus on the particular context of an individual school district and the meaning-making of its actors, this study embarked on a form of policy analysis called for in recent educational policy literature

(Anderson, 1987, 1989, 1990; Foster, 1989; McLaughlin,

1987; Miller & Lieberman, 1988; Timar St Kirp, 1989;

Wimpelberg, Teddlie, & Stringfield, 1989). This form of policy analysis is concerned with how the macro aspects of policy made at one level interact with the micro realities of administrators and teachers within unique, local organizational contexts. As McLaughlin (1987) describes this new form of policy analysis: "The problem for analysts comprises linking the nominalistic world of the Btreet level bureaucrat to the systemic patterns that comprise the world of policymakers" (p. 177). By focusing on the situations where macro policies meet micro contextual realities, research can shed light on the organizational and individual aspectB of policy implementation. 347

Although previous policy implementation studies have

focused on school reform legislation initiated in the last

decade (Chance, 1988; Darling-Hammond &. Berry, 1988;

Ginsberg & Berry, 1990; Mitchell & Encarnation, 1984;

Murphy, 1980; Orlich, 1985, 1989; Timar & Kirp, 1988,

1989), little attention has been given to how these

policies are interpreted and enacted by local educators and

to the implications of the policies for redefining the

organizational reality of the local context. With the hope

that such studies can create new understandings regarding

what types of policies might best support educational

improvement, this inquiry examined the influence of a state

testing policy on the work and relationships of local

educators. With the belief that discourse is the primary

means through which human actors create, sustain and

potentially change their constructed social reality, the

investigation centered on the conversations of educators

about a new state testing policy.

The testing policy required school districts to

administer achievement and ability tests selected from a

state approved list to students in grades 4, 6, and 8. It

also required ninth and twelfth graders to take a state developed "proficiency" test. Student performance on the

statewide proficiency tests affected whether they would

receive a diploma (as opposed to a Certificate of

Attendance) from their local high school and the nature of 348 the diploma they would receive (Diploma of Basic

Competence, Diploma with Distinction, or a Diploma with

Commendation).

The school district of interest ("Foster") is in a rapidly growing city in large metropolitan area in the

Midwest. The previous small farming community iB being rapidly transformed into a fairly affluent city. Both the community and the school district are feeling the effects of unrelenting and unprecedented growth. The school district has been able to select new teachers and administrators from thousands of applicants and has tried to develop a culture of professionalism in which these educators can act on their professional knowledge base in making decisions regarding the curriculum and instruction of students. In general, the organizational culture of

Foster is characterized by havens of supportive relationships for professional practice among teachers and administrators within schools.

Equivalent professional havens appear to be lacking across schools, however. There seems to be a lack of district-wide identification and supportive relationships among building prinicipals. Among some board members, and between some community and board members, the relationships are more problematic, characterized by conflict, tension and controversy over board decisions. In addition, some members of the community are becoming critical of the 349 continued increases in school taxes accompanying the rapid growth of the district.

The findings from this inquiry into the initiation of the state testing policy into this local context are summarized below in terms of the specific research questions.

Summary of the Findings

1. How did the state testing policy relate to the conversations in the professional culture of local educators in this organization?

a. How did educators make sense of the policy, including their initial reaction to it?

b. How did educators see the policy influencing their work and relationships?

c. What rules for discourse Bhaped the state testing policy?

d. What rules for discourse shaped the professional culture of local educators?

The Foster educators' initial reaction to the state testing policy included a strong sense of ambivalence. On the one hand, they recognized the state's legitimate right to be concerned about education and to hold them accountable for the educational well-being of students. On the other hand, they were concerned that the selected measures for accountability, achievement and proficiency tests, did not match their local vision of what schooling should be nor the content of their written curriculum and 350 daily instructional practices. The tests’ emphasiB on isolated skills and workbook page formats contrasted with their emphasis on application and integration of complex skills. In effect, they felt the tests reflected an archaic way of thinking about how children learn.

Furthermore, Foster educators were concerned with how the results from the achievement teBts would be used.

Although they believed that the tests did not match their content or instructional approaches, they feared that the results would be used to unfairly compare districts, schools, and teachers. Given that the tests could only measure a very narrow aspect of what education is about, and given that students come to school with different strengths and different needs, Foster educators felt judging the quality of schools or teachers on the basis of test scores was misguided and unfair.

In addition, the comparisons engendered by such judgements would increase competition among educators. In contrast to state policy makers, these educators saw competition, not as a spur to improve schooling, but as dysfunctional to the central purpose of educating all students. Collaboration and cooperation among educators, rather than competition, were seen as necessary qualities for improving schools. In keeping with a culture of professionalism in which all members contribute their different talents for the benefit of all students, Foster 351

educators foresaw no improvement coining from a policy which

placed one school or one teacher in competition with

another.

Another concern of Foster teachers and administrators

was the resources funneled into meeting the state testing

requirement. The tests consumed considerable district funds

as well as time, energy and attention of administrators,

teachers, and support staff to prepare for, administer, and

analyze the results. The financial and human resources

expended on these tests, which were seen as having

questionable instructional value, were therefore not

available for the development of assessments which would

match the local curriculum and which could be useful

accountability measures.

The mandated testing created a professional dilemma for

Foster educators. They felt they had to choose between

doing what their professional judgement said was best for

the education of students and doing what would make the

test scores increase and thereby make themselves and their

schools "look good" to constituents. This dilemma was

especially acute for administrators who were in the

position of enforcing a policy with which they disagreed, at least in part. Having to ignore their own professional

judgement, as well as that of teachers, in order to

implement the testing policy was difficult for administrators who were simultaneously trying to promote a 352 professional culture which depended upon the daily exercise of educator judgement.

Although Foster educators stated that they did not want to "teach to the test", the indication from the elementary data suggested that teachers who were familiar with the achievement test content from previous yearB did in fact insure that their students received instruction on the content and format of the questions prior to the test administration date. Although the local curriculum guide and their own instructional practices may have suggested, for example, that fractions be taught later in the year or with an instructional approach that did not rely on multiple choice formats, teachers who knew that their students would be given questions on fractions included this instruction prior to the testing date.

In addition, teachers and administrators involved in the revision of the language arts/reading curriculum, as well as those reconsidering the secondary mathematics program continue to deliberate on how much their written course of study documents and daily instructional practices should be guided by the nature of the state required achievement and proficiency tests.

Thus, in their initial reaction, Foster educators viewed the state testing policy with apprehension, not denying the need to be accountable, but not agreeing with the state's approach to accountability. The effect of the 353 policy on their work already could be seen by the adjustments being made in their instructional practices and their deliberations over adjusting their written curricula in order to align them with the content and format of the achievement and proficiency tests. Foster educators feared that the use of test scores to compare schools and districts would eventually have an adverse affect on relationships among educators. Responding to the testing mandate absorbed local resources which were then unavailable for the development of alternative accountability measures. Foster educators were thus in the position of objecting to the current manner of being held accountable, without having an alternative to purpose.

The conversations regarding the state testing policy were shaped by two contrasting sets of discourse rules. The discourse rules inherent in the policy required unquestioning compliance of educators who were accustomed, within their local organizational culture, to participating in professional decisions regarding their curricular and instructional practices. Furthermore, as noted above, the policy called for using the test scores as a basis for comparing the quality of schools and districts. Educators believed this was a misuse of test information and tried to avoid such comparisons in their conversations about the test results. Given the conflicting setB of discourse rules regarding how to talk about the testing policy, coupled 354 with a climate of increasing criticism and controversy in the district, it was difficult for administrators to articulate their particular vision of schooling and to explain the inappropriateness of the state testing measures as indicators of success for their local instructional practices.

2. How did the research process relate to the conversations in the professional culture of this organization?

a. What rules for discourse shaped the research process in this organization?

b. How did the researcher influence the individual conversations about the state testing policy?

c. How was the researcher influenced by conversations with the educators in this organization?

The discourse rules inherent in the research process included the engagement of the researcher in active dialogue with the participants. Research-related conversations helped individual practitioners clarify their thinking about the state testing policy and helped the assistant superintendent, who served as the key informant, reflect on his practice. These conversations also drew the researcher into the some of the work of the practitioners.

I helped the assistant superintendent understand the technical aspects of the achievement test results and helped him prepare his presentation of the results to the local school board in a way which avoided comparisons being 355

made among schools. In addition, I became committed to

helping these educators articulate their approach to

schooling and develop appropriate means for assessing their

local practices.

3. How did the state testing policy and the research process combine to bring about new conversations among educators in this organization?

a. How might local educators reframe the way in which the state policy holds them accountable?

b. Can the research process be a catalyst for collective conversation regarding how educators are being held accountable?

c. What organizational conditions facilitate the start of new conversations?

Conversations about the testing policy and the research process provided local educators with occasions to express

their concern over potential ill effects of the policy.

These expressions were often conveyed in passionate and eloquent language. I recorded their expressions and presented their words back to them in an attempt to use the research process as a springboard for new conversations and new action. The written member checks were intended to demonstrate the commonality of views across administrators who had said they felt alone in their perspectives. It also might have served as a catalyst to prompt joint action on articulating their view of education and developing an alternative method of being held accountable within their vision of what the educational process should be. 356

A number of reasons were proposed for why these new conversations did not take place: the demands on educators time; the tangential nature of the research to their work; the continued arrival of new students in the face of budget constraints; and the fact that the topic of testing policy was not among the priority items for the superintendent.

In addition, I suggested that the absence of strong and supportive district-wide relationships among administrators and board members may have prevented their coming together on the issue of how they should be held accountable.

Ironically, it seemed that at the time when local educators most needed to articulate their success at schooling to their local constituents, they seemed least able to come together to discuss how this might be done. I wondered if in order to engage in the kind of conversations necessary to collaborativly construct new ways of evaluating their performance, practitioners might need to have already established relationships in which they feel assured that they will be listened to, that their emerging ideas have value, and that together they can create something new and worthwhile. Without trusting relationships to provide the setting for these interactions, such collaborative discourse about their vision of education and an appropriate means to assess it may not be possible. 357

Research Conclusions and Discussion of Implications

In this section the research conclusions are discussed

in terms of three implications from the study: 1) the

impact of state testing policy on local professional

culture; 2} the role of discourse in creating

organizational possibilities; and 3) methodological

quandaries related to attempts at participatory research.

Vulnerability of the Professional Culture

One conclusion from this study pertains to the fragile

nature of a professional culture in the face of a clash

between two approaches to school reform. In effect, the

state testing policy represents an attempt to improve

schooling by mandating that educators align their

curricular and instructional practices to the content and

format included on state required tests. Coupled with the

intent to compare Bchools on the basis of the test results,

policy makers across the country see tests as mechanisms to motivate educators to act in certain ways. Rewards are given to schools and students who do well on the tests and

sanctions are imposed for those who do not. The "high

stakes" associated with test results are intended to motivate educators to act in a way which will reform

schools. The Ohio state testing policy is an example of

this intent to motivate action. By providing educators with

information on student needs, and by providing parents and 358 other taxpayers with comparable information about the performance of students, the norm-referenced achievement tests and state developed proficiency tests were intended as a source of useful data for school improvement. Teachers and administrators were expected to align their curriculum and instruction with the content and form of the test, and in so doing, improve the performance of students.

A contrasting approach to school reform focuses on the development of the professional capacity of educators to make wise decisions regarding the curriculum and instruction of students. Based on the definition of professionalism offered by Darling-Hammond &. Berry (1988),

"members sharing a common body of knowledge and using shared standards of practice in exercising that knowledge on behalf of clients" (p. v), the actions of members in a professional culture are characterized by specialized knowledge, self-regulation, special attention to the unique needs of clients, autonomous performance, and responsibility for client welfare. In all occupations which claim to be professional, the right to decide which actions to take exists "in some tension with alternative forms of regulation and accountability, with continual adjustments made on all sideB to enhance the public good" (pp. v-vi).

In effect, any profession is caught in the tension between having its members decide upon their actions and having these actions regulated by some outside body. How 359 much the state testing policy should regulate the curricular and instructional practices of educators versus how much these decisions should be left to local educators is the issue at stake in this study's example of a school reform effort.

School reform based on professionalism would emphasize the improved capability of those closest to the student to make wise curricular and instructional choices on behalf of the student. The strategy of increasing educator capacity is based on the premise that what matters most in education is the quality of the interactions between teachers and students. In essence, the quality of teacher-student interactions is critical to the quality of schooling and the nature of these interactions cannot be improved by external regulations. A number of scholars argue that since teachers make the most important discretionary choices in education, wise state policy would maximize teacher control over the instructional process. The way to improve instruction is therefore to improve teachers, rather than to have policy makers tell administrators to tell teachers what to do. This alternative relies on the establishment and maintenance of a professional culture among educators where teachers accept and relish the responsibility for making curricular and instructional decisions (Boyd, 1987;

Conley, 1988; Darling-Hammond, 1988; Elmore, 1983, 1987;

Elmore & McLaughlin, 1982, 1983; McLaughlin, 1987; McNeil, 360

1988b; Shulmanf 1983; and Wise, 1979).

Given these two contrasting views on how to improve

schools, what impact does Ohio's state testing policy have

on the district of Foster? On the one hand, state

legislators see the policy as improving the information

that educators have about students. Educators in Foster,

however, see the state test information as redundant to

better information available from classroom assessments and

as somewhat misleading in its lack of fit with their local

curricular and instructional emphasis. On the one hand,

state legislators see test information as important to the

legitimate right of parents to compare schools. Educators,

on the other hand, see test scores as very narrow measures of what schooling sets out to accomplish, and the

competition resulting from such comparisons as harming

rather than helping the education of all students.

In my view, this study illustrates that a state testing

policy, although it does not have the power to insure

quality interactions between students and teachers, has the

potential to challenge the professional culture of a school

district. By requiring that educators be judged with what

they perceive as inappropriate accountability measures, the

state testing policy can shake their vision of what

education Bhould be and their confidence in their

professional judgment, and thereby potentially undermine

their capacity to respond to the needs of their students. 361

The state testing policy has the potential to challenge

what Corbett et al. referred to as the "sacred norms" of

the Foster organization, that is, the norms that define the

identity of the members:

Attacks on the sacred undermine professional identity and call the meaning of teaching into question. Proposed changes, then, challenge more than "the way we do things around here" [profane norms]; they also threaten "who we are around here" [sacred norms]. (1987, p. 56)

For Foster educators the state testing policy challenges their definition of what the local curriculum

and the local form of instruction should be, and what they define as "best for students". If the state takes control

over this definition, educators would be denied the

opportunity to exercise their body of knowledge in making decisions on behalf of students; in other words, they would be denied the right to act as professionals.

The conflict over whether the state or local professionals should control curriculum and instruction can be seen as no less than a conflict over authority in

schools. Who has the authority to decide what the curricular and instructional focus of schools should be?

Should legislators make this decision by mandating particular tests which become the outcome measures for

schools? Or should this be the prerogative of local

educators who work daily with particular students and claim 362 to know student needs and the wishes of the local community better than any state legislator could?

Elmore (1987) argued that the answer to this question is essential to the learning process and therefore critical to any attempt to reform education. He asserted that the character of a school is dependent about how authority is constructed between teachers and students. Among the factors influencing the construction of authority in each school are teachers' predispositions to influence students' learning and students' predispositions toward learning and toward accepting adult influence. That is, authority in schools is a reciprocal relationship, including a granting of legitimacy by the students to the teacher. If students do not consent to adult influence and/or adults are not willing to or do not have the skill to exercise authority on behalf of student learning, little learning will occur.

If the learning process requires the consent of the students to be influenced by teachers and the willingness of teachers to exercise their authority on behalf of students, what role do external mandates play in this interaction? Elmore maintained that regulations from outBide the teaching/learning process which attempt to control the content and form of teaching interfere with and undermine the legitimate authority relations between teachers and students: 363

External mandates may help in minimal learning conditions, but they seldom, if ever, reinforce students' predispositions to accept adult influence, or help adults exercise that influence. In fact external mandates may destroy interaction between adults and students in schools. ...They force adults in schools to decide, implicitly or explicitly, whether to consent to the legal-rational authority of those mandates. If they consent, they become agents not only of those mandates but of those who set the mandates, thereby losing some portion of classroom authority, "I don't make the rules, I just enforce them," is the classic formulation of this posture, (pp. 70-71)

If students view teachers as functionaries, carrying out the orders of external others, they are unlikely to submit to the authority of teachers. Elmore argued that if teachers view themselves as agents of the state, simply there to pass on packaged knowledge but without a role in the creation of knowledge or choice of what knowledge is important to teach, they are unlikely to exercise their skill on behalf of students. Rather than improve schools, external mandates which undermine teacher-Btudent authority relations can thus make the situation worse.

In effect, such regulations cannot mandate what matters; that is, they cannot guarantee that the interactions between student and teachers will be of high quality, will be expressions of teacher intellectual and emotional investment in student learning, and student acceptance of a teacher's right to influence the student's thoughts and values. 364

Another way to look at the struggle over who defines the nature of the school’s curriculum and instruction is to see it as a conflict over who controls the definition of knowledge for schools. Basil Bernstein identified the importance of this struggle in hiB 1971 analysis of schools and society. He identified three "message systems" in schools, one concerned with what knowledge is, another with how it should be transmitted, and the third with how it should be assessed:

Formal educational knowledge can be considered to be realized through three message systems: curriculum, pedagogy, and evaluation. Curriculum defines what counts as valid knowledge, pedagogy defines what counts as the valid transmission of knowledge, and evaluation defines what counts as a valid realization of this knowledge on the part of the taught. (Bernstein, 1971, p. 47)

In their expressions about the state testing policy,

Foster educators claimed that they should have some say in what counts as valid knowledge, what counts as valid instructional practices to convey that knowledge, and what counts as valid assessments of what students have learned.

In this struggle over the definition of knowledge they have tried to retain some control over education in Foster. The alternative, in their view, would be to relinquish control to the state definitions of curriculum, instruction, and assessment implied in the state mandated tests. 365

Thus determining how much influence the state testing policy will have on the curriculum and instruction in

Foster includes struggles over professional identity, over the construction of authority relations between teachers and students, and over who controls education in terms of deciding what knowledge is important and what form the

transmission and evaluation of that knowledge shall take.

Such struggles represent the difficulty of balancing

the needs for educator autonomy and accountability. The public has a right to insure that its children are educated

in an effective, just, and equitable manner. The central dilemma is how can policy support conditions in which

educators are both free and responsible (Shulman, 1983).

That is, how can teachers be free to flexibly respond to

the diverse needs of the students in front of them and yet

assure the public that they are responsibly acting in a manner that promotes learning for all children? The search

for the right mix of top-down mandates and bottom-up

educator capacity-building continues.

In Ohio, one could argue that the state testing policy

is appropriate as a top-down mandate because it serves as a broad educational goal but does not tell educators how to

reach the goal, thereby allowing them to determine the

strategy as part of exercising their professional

expertise. In Foster, however, educators found the state

testing policy as intruding into professional decision 366 making because it precluded the most important educational decision: what knowledge is important, and it indirectly

implied a teaching method: focus on questions which have

one right answer; which can be answered with paper and

pencil; which can be chosen from a list of alternatives;

which are best answered in silent, timed conditions and

without clarifying help from teachers. To Foster educators,

the state testing policy implied that only those questions

lending themselves to this very limited instructional

format were worthy of educators’ and students’ time, energy and attention. This was an implication with which they, in

their professional judgement, disagreed. In their minds, the state testing policy worked against their attempts to be both responsible and free in their work as educators.

Moreover, the impact of the state testing policy in

Foster was additionally influenced by the particular nature of the Foster organizational context. Given that the district was made up of havens of professional support within schools but not across schools, the state testing policy spotlighted a vulnerable aspect of the organization: the lack of a district-wide consensus on how they should be held accountable. In other words, the impact of the state

testing policy may have been more keenly felt in this district because it touched on the weak link across

schools. It challenged the professional culture because it 367 asked the question of accountability about the aggregate, that is, about the students and schools as a whole.

The indicator of success was removed from assessments of individual students in individual classrooms and focused on an assessment of the collective. Given that the indicators of success required by the state testing policy

(achievement and proficiency test scores) did not match the instruction Foster educators have offered, and that these indicators assumed a form of schooling which was in opposition to their professional vision of how to educate children, they were left without accountability measures which both they and the local community recognized as legitimate indicators of the quality of their schools.

In addition, the state testing policy insisted on comparing schools with one another based on the test results. The ensuing competition placed the well-being of one school against that of another and reinforced the tendency to identify with and protect one's own building.

Without supportive and trusting relationships among principals, central office administrators, and board members, it may not be possible for these actors to engage in the the conversations necessary to articulate a more appropriate measure of success for their schools. Without a collectively defined alternative to the state testing policy, the policy can add to the discontent and strife characterizing the relationships among the school board, 368 administrators, and community, and thereby undermine the professional culture educators have tried to create within their organization.

As a result, teachers may do even more "teaching to the test" despite their professional belief that such practice is not in the best interest of children. As noted in this study, state imposed achievement testing affected what was taught in the classroom, whether or not the content was included prior to the testing or at all in the local curriculum. Teachers who knew what was on the test from past experience insured that its content and the format in which the content was presented was taught in their classrooms. Although this may have been an unconscious choice, or a conscious but unspoken choice, the overall consequence was that the test did influence the "lived" curriculum in the classroom.

This example of Argyris & Schon's (1978) distinction between espoused theory and theory-in-use, shows the power of the state policy to effect local practice. The espoused theory of these educators was that they believed in the local curriculum and used it, not the content and skills of the state mandated test, to guide their instructional practices. Their theory-in-use, however, took into account the high political stakes associated with state testing, and insured that their students received instruction on the content included in the test prior to the test 369 administration. Since they as teachers, as well as their students, their school, and their district could "look bad" if the scores are low, the curriculum and instruction were altered to insure that students had every chance to score highly on the test.

In conclusion, the state testing policy seemed to have had an adverse effect on the professional culture within

Foster schools. Although this conclusion may not hold for other Ohio school districts, it seemed that in this district where there had been an ongoing and somewhat successful effort to establish and maintain a culture for educators to act as professionals, the testing policy challenged that culture by wrestling professional decisions over curriculum, instruction, and the authentic assessment of instruction away from educators. In other words, for this district, the approach to school reform which relied on state regulation of what knowledge gets tested adversely affected the approach to school reform which relied on the maintenance of a professional culture. Coupled with the specific organizational context which precluded these local educators from mounting an alternative way of being held accountable, the state testing policy weakened the Foster educators* capacity to decide upon and articulate their vision of educating Foster students. 370

Discourse and Organizational Possibilities

The struggle over control of education in Foster was manifested in the struggle over which set of discourse rules will dominate. The discourse rules from the state policy asserted top-down hierarchical decision making with comparisons and competition as the focus for conversations about school reform. The discourse rules from the culture of professionalism asserted bottom-up shared decision making and educator collaboration as the conversational focus for organizational change. This struggle was seen in the dialogue between administrators and the local board.

For example, when the assistant superintendent brushed past a board member's questions about the need for suburban norms to interpret the achievement test results in fear that the next question would be about comparing schools on the basis of the test scores, he was trying to assert the discourse rules of the culture over those of the policy. As

Wilden (1987) argued:

The dominant formB of communication in a given society (both verbal and nonverbal) define and constrain its ways of seeing and believing, and knowing and judging. They form the ground of what the dominant members of the society accept as true and false, legal and illegal, legitimate and illegitimate, (p. 132)

The fact that educators saw comparisons based on test scores as illegitimate while policy makers saw them as legitimate set up a struggle for which rules of discourse 371 would dominate their conversations; that is, which group would control the situation. Within Foster such struggles signaled that something was amiss in the interactions between the two groups. There was an underlying difference in how the local policy makers (who were adopting the discourse rules of the state policy makers) and the local educators were viewing education.

This conflict could have provided an opportunity for organizational change. As Deetz & Kersten attested, contradictions in an organization are "important internal mechanisms for change" (p. 166). Because they point out limitations in the present system, contradictions can offer the possibility to change the system. In Foster, the conflict over how to use the scores emanating from the state testing policy could have opened up the possibility for creating a new understanding about schooling among all the interested parties. Rather than have the educators operate with one image of schooling and their local policy makers and public operate with another, the conversations surrounding the state testing policy could have promoted the creation of a shared image.

The potential for such discourse partially depends upon whether educators will engage in a language of choice.

Their current language conveyed a mixture of fatalism and defiance. For example, at one point in the local talk about testing, the assistant superintendent said, "We have no 372

alternative but to teach to the test". In contrast, during

another conversation an administrator said to me, "I’d like

to be the first district in the Btate to stand up and say

'We are not testing, period*". During a different

conversation another administrator proclaimed, "We are not

good at telling people *NO’, we’re not going to do

[testing] and these are the reasons that we are not going

to do that, and take us to the wall, but don’t expect us to do something that we know is not good for kids".

Whether such strong feelings are heard outside of private interviews with the researcher Bomewhat depends upon how educators see their standing in the local context.

Is it politically feasible "in a community of lawyers" not to have students Bcore well enough on the state proficiency test to obtain the highest level diploma? Could educators adequately explain why the criteria for the diploma

(specific scores on the twelfth grade test) were not important or suitable accountability measures for the educational program within Foster? It would take a strong leap of faith for local educators to ignore the political implications of their students scoring poorly on the state test.

Perhaps my expectation that the research process could promote change-oriented discourse in Foster was unrealistic. These educators, overwhelmed by the constant arrival of new students and drained by community criticism 373

and strife, night have been incapable at this point in time

of "iraaginizing" (Morgan, 1986) a different organizational

reality. They may not be able to visualize district-wide assessment/accountability devices which would match their daily instructional program. Given that they had not already clearly articulated their image of schooling to their local public, it may not have been possible for them

to communicate this vision and to simultaneously create an alternative way to be accountable for it while they were in the midst of local turmoil and while the state was requiring them to be accountable according to state determined indicators. Such an effort might be analogous to trying to rebuild a ship while at sea (White, 1979).

The question then becomes what conditions are necessary for a language of organizational change and transformation to occur? From this study, it seems that for Foster to engage in collaborative discourse about their vision of education both among educators and with their local community they needed a clear message from the organizational leadership that such a discourse was an organizational priority. Administrators also needed to have established strong professional relationships across buildings and to have identified what the district-wide

"we" stands for in Foster. They then needed to become articulate converaers regarding their district-wide vision of education in Foster. 374

This is no easy task, however, and its difficulty is related to the inherent ambiguity of the whole education enterprise. As explained by Deal and Wiske (1983):

If teaching or managing schools were certain, clear and straightforward tasks, then educators could find a haven in a professional culture or technology. But education is an indeterminate enterprise. Its purposes and technologies are unclear. Its goals are diverse, diffuse, and disputed among various stakeholders. Why and how students learn - or if they do at all - is hard to define, difficult to measure, and unlikely to be disentangled from multiple events that contribute to student growth and development. For all these reasons, teachers and administrators have a hard time knowing, demonstrating and proving their effectiveness. As doubts and accusations mount from a public that has lost faith in schools, educators are strapped for responses. Their ambiguous answers reflect the reality of what they do; their self-doubts increase as the public presses for more certainty in a process that is inherently ambiguous, (p. 452)

The Foster educators are strapped for responses when confronted with the demands for accountability inherent in the state testing policy. They know the form of measurement required by the state does not fit their educational enterprise but they are tongue-tied in their efforts to explain why. They need to develop the competence to effectively communicate what it is they do in their schools.

One aspect of communicative competence is the use of metaphors. Foster educators may need to develop metaphors with which they can clearly convey the difference between 375 i their instructional program and that implied in multiple-choice tests. It was interesting to note that

Foster educators used metaphors to explain their understanding of state policy makers' images of education.

That is, they objected to the industrial, factory, business, and military metaphors they attributed to policy makers. They did not, however, have metaphors to express

their own alternative image of schools. The closest they came was one administrator's declaration that "We are not a business, we are learning centers for people". The development of metaphors to identify and convey their vision of schooling may be an important first step in establishing a discourse of possibilities in Foster.

In conclusion, the discourse over testing displays the

struggle over who will control curriculum, instruction and evaluation in schools. For educational professionals to persuade local, and perhaps state, policymakers to accept their view of the schooling enterprise, they need to develop the communicative competence to articulate and convincingly convey their image of what education is and how it might be appropriately assessed.

Implications of Participatory Research

There may be a role for researchers to play in the process of developing the communicative competence among educators. This study found that the research process could 376 help practitioners clarify their thinking about the Btate testing policy; it helped to articulate the "unfocused and uneasy malaise about testing", as put by one administrator,

The research process can also help practitioner’s reflect on their own practice and to ask them why it is that they make sense of a phenomenon as they do. With this type of researcher involvement in the ongoing dialogue of the organization, research can help lay the groundwork for practitioners to express their vision of education (Deetz &

Kersten, 1983) ,

This engagement of the researcher is not without problems, however. The questions regarding researcher bias and my alignment with the perspectives of the participants raised in thiB study point to some of the dilemmas in participatory research. On the one hand, my involvement in the conversations and work of the district could have co-opted my independent perspective regarding Foster educators and the effects of the state testing policy on them. On the other hand, without being completely engaged in the conversations, without disclosing my values and perspective, without establishing authentic human relationships with the people I was studying, I do not think I would have been given access to their sense-making.

It would have been reasonable for them to be very careful of what they said to me so that my words about them would not do them harm. In our jointly written article, Bill 377 described that as he got to know me and came to trust my regard for educators, he gave me wider access to sessions and people in the district. If I had maintained a more traditional researcher distance I may not have had access to the data of interest to the study.

Yet, as I became known to them and they became known to me, I became convinced that the state testing policy could adversely affect their decisions about curriculum and instruction and their attempts to maintain a professional culture. This development could have clouded my critical eye. I might not have taken advantage of occasions when I could have argued for the merits of the state policy. Also, during the unfolding of the study, as part of the discussion with my dissertation committee members, we began to wonder if the research process might help Foster educators reframe the accountability issue. As recounted in

Story 3, I did make this attempt, and although I still believe it was an important effort, I now see it also moved me toward seeing myself a3 "one of them".

In considering whether I was co-opted into seeing only their view, one line of argument might be that my involvement was in response to their stated need. That is,

I did not impose an "anti-test" view on them. They expressed their concerns with the tests, and then expressed their dilemma between not resisting the need to be accountable, but resisting this form of accountability. I 378

asked in response, can the research process help with this dilemma? That is, can the research process Berve the

"subjects" of the research? Is the use of the research process to serve the participants the equivalent of co-option? I have no certain answer, although I do not think so.

I have come to see this quandry as part of a larger distinction which Emerson made to the faculty of Harvard

College in 1837: the development of intellect and the development of character are not the same. My translation of this statement tells me researcher knowledge i3 different from practitioner responsibility. Can a researcher ethically and authentically enter the practitioner context, deal with an issue that is important to them, and not get involved in their work? The collaborative researchers and action scientists would say no. The canons for how to do this type of inquiry, however, while maintaining a critical stance on how it is that they

(not we) are socially constructing their reality, are not yet established.

I have come to wonder if it is possible for the researcher to live as a permanent "devil*s advocate" in the midst of practitioners or whether the researcher must move in and out of this role during the research process. In this study Bill revealed that at the point in the study when I started to help him with the achievement test data 379 he stopped feeling like a bug on the slide of my research microscope:

Only at this point did I as the person under constant scrutiny feel I was coming out from under the microscope and thus changing the researcher/subject relationship.

If moving out from under the researcher microscope and

involving the researcher in practitioner work represents a positive change, what is the "dark side" of researcher involvement? Does it present an obligation for practitioners to go along with the researcher’s line of thinking and questioning? If so, is this impositional and exploitive? What if the researcher proposes a very impractical and perhaps dysfunctional action? Does the fact that it comes from an outside researcher give it more weight in the deliberations; or will it be easily dismissed as a suggestion from someone who doesn’t "know" what local participants know? I do not have answers to these questions yet I feel that the answers will vary with the context and the personalities of the researchers and the practitioners as well as with the substantive issues being addressed in each inquiry.

Perhaps a middle ground is for the researcher to adopt a stance of being responsible to practitioner work, but not for practitioner work. That is, the researcher cannot do the work him or herself, but the researcher can engage in the issues of the work, be present to it, be interested in 380

it, play the role of helping practitioners articulate, understand, and consider alternatives to decisions, rather than just stand back, watch, and report on practitioner decisions.

The final conclusion I draw from this study is a methodological and political one: It is easy for the researcher to enter a site seeking academic knowledge; it is also easy to hide under the researcher's methodological hat and take no responsibility for affecting the life that is found there. Collaborative research, however, seems to carry with it a responsibility toward the work of the people in situ, a responsibility that cannot be shrugged off with a wave of the researcher’s hat.

Continued exploration of scientifically valid and ethically appropriate ways for researchers to share in the responsibility to practitioners' work is one of the implications for further study emerging from this inquiry.

Implications for Further Study

As noted in the introduction to this inquiry, there were many conversations in the school district that I did not follow in the search for how the state testing policy was being implemented. A future, perhaps multidisiplinary, team of researchers may want to explore in unison some of the following conversations as they pursue the impact of 381 state policy on local practice:

- political conversations: for example, follow how the policy threads through thepolitical culture from the state legislature, the state board of education, the state department of education, to the local board of education, the superintendent, the teacher’s union, etc.

~ instructional conversations: for example, follow how the policy is talked of among teachers, and most especially, between teachers and students. Observe how it is enacted in the classroom practices of teachers as they interact with andgive assignments to students.

- community conversations: for example, follow the conversations between teachers and parents regarding the test scores of individual students, or between school board and community members regarding the district-wide scores, or across districts as the state requirement to publish the scores by district and by school goes into effect. Does such publication result in parents comparing schools on the basis of test scores? If so, what is the nature of these conversations?

Additional inquiries into the effect of state testing policy might focus on the macro historical, economic, political and cultural forces more common to critical science. For example, an historical/economic question related to this Btudy could be how is it that educational funding has developed in such a way that a district such as

Foster has enough resources to attract applications from thousands of educators, while other districts go begging for qualified teachers and administrators. Whether or not schools need to have a certain economic base for the professionalization approach to school reform to be viable could be a subquestion in such a study. That is, how much 382

does Foster's overall economic status have to do with its

ability to establish and sustain a professional culture

among educators? Would other less financially supported

districts, such as those in urban centers or in rural

areas, be able to attract the type of personnel needed to

create a culture of professionalism? If not, is this one

more illustration of the inequities inherent in the

schooling system in the United States?

A final suggestion for additional research pertains to

the understanding of organizations as people in

relationships. Additional inquiries might pursue the

conclusion from this study that the lack of supportive,

trusting relationships among administrators across schools coupled with the strife between board members hampered

educators' ability to articulate a district-wide statement of their educational vision and a district-wide alternative

to being held accountable with state designated tests.

Whether participants have to establish trusting personal

relationships with one another before they can engage in collaborative discourse could be studied in a variety of contexts. The first step in their search for ways to

improve schools may be for policy makers and educators to

establish trusting relationships with one another. Once

these relationships are established, they may be able to

jointly construct new knowledge and new ways of acting in

schools and on behalf of children. APPENDIX A

DESCRIPTION OF PROPOSED RESEARCH STUDY

383 Manor Zajano » OSU 292-7700 2-2-89 Oeaoription of Proposed Research Study

Mr raaearob interest la on how state polier influenoes the work of administrators in aohool districts, espeoiallr that of school principals. The newly mandated state testing polior (both the aohievesent/abilitr testing in Oradas 4, 6, and 8* and the profioienor testing of ninth graders) ia an example of a state polier which has great implications for local educators, especially administrators. I would like to follow this polior through its implementation in rour school district, noting how it affeots the deoiaions, attention, priorities and relationships of principals, and also of teachers and other administrators. Because the work of the prinoipal is embedded in the larger context of the whole sohool srstea, and beosuse state policy comes to the principal via the oentral office, I would like to include a study of the larger context as a part of my focus on the prinoipal. I anticipate: 1. Attending oommittee meetings (taking notes and/or tape recording them if permitted); 2. Interviewing principals, other administrators, snd teaohers who osn shed light on how this particular state polioy influenoes their work; 3. Reviewing looal documents that pertain to the lmplesentation of the state polioy (e.g., committee memos) If given permission by a particular principal: 4. Extensive shadowing and interviewing to see how he/sbe interprets state poliaiea and their effect on his/her work and relationships. This work will be summarised in written foru in a Ph.O dissertation. Anonymity of the distriot and of individuals will be respeoted. Mo one will be Interviewed or shadowed without their individual permission. Opinions expressed will be held in oonfidenoe.

About myself: Former teaoher; 8th grade, Chioago Public Sahools Program evaluator: Wisconsin and Rhode Island; (evaluation of national, state and locally funded programs); Currently doatoral student in Educational Polioy and Leadership, OSU; Graduate Researoh Assooiate with PROBE (Polioy Research for Ohio Based Education) APPENDIX B

FOSTER RESOURCES USED FOR STATE MANDATED

ACHIEVEMENT AND ABILITY TESTING

385 386

Time, Energy and Attention of Foster Personnel Expended to Conduct State Mandated Achievement and Ability Testing, Spring 1989

1. Central office administrators:

-Fall 1988: Testing committee visited local school districts to see their solution to hardware/software decisions (8 central office administrators served on this committee);

-Dec.'88-Jan.'89: Meetings with test publishers, hardware and software vendors, school board, and treasurer prior to choice of tests, and choice of hardware and software to scan, score and analyze results,

-January: Review CAT test items to compare with local courses of study. - Selection of test levels to order

-January - May: Ongoing work with vendors to get hardware and software working with local data.

-January - March: Input previous CAT data to develop student identification system and to test accuracy of software/hardware scoring

-March: Prepare content and materials for teacher inservice sessions.

-April: Prepare for and conduct 6 inservice sessions regarding CAT and TSC for teachers of grades 3, 4, 6, and 8. - Arrange for student names and other identification information (e.g., sex, birthdates) to be put on answer forms and "bubbled in" prior to distributing to schools. - Distribute testing materials to 9 schools. - Collect tests. - Check and edit answer sheets for unacceptable markings prior to scanning and scoring. - Develop drafts of test reporting formats. 387

-May: - Scan test answer booklets and answer sheets. - Score tests - Analyze results by student, class, building and district - Develop score reports for parents, teachers, and principals - Return results to parents and principals. - Prepare summary tables for school board and general public - Answer questions from principals, teachers and parents regarding specific scores. - Do additional analysis for specific questions from principals

Central Office administrators overall time: For the assistant superintendent, these efforts absorbed a minimum of 20-30 hours per month from January to June. For the computer education specialist these efforts used an estimated 25% of time from January to mid-March and 80% from mid-March through May. Other central office administrators also were involved but for briefer portions of time.

2. Building Principals / Guidance Counselors:

- Notify students and parents of testing dates. - Change schedule for all classes both in the morning (when testing is going on) and in the afternoon (to accommodate for missed classes in the mornings). - Make special arrangements for learning disabled and English as a Second Language students. - Prepare and pass out all testing materials to classrooms. - Arrange to give all students a break at the same time so as not to disturb each other and to standardize the testing conditions. - Make note of students absent during testing. - Schedule and complete makeup testing for absent students. - Rearrange teacher planning time as a result of changed student schedule. - Insure that no announcements or bells interrupt the testing. - Develop and place signs on doors indicating that testing is taking place. - Arrange for completed tests to be organized and picked up. 388

- Answer Central Office questions during scoring process - Review test results with relevant groups of teachers. - Send scores to parent(s) of each student. - Respond to parent inquiries regarding scores. {As noted earlier, one principal Bpent several weeks on this task during the school year and planned to spend more time reviewing scores during the summer.)

3. Teachers:

- Attend inservice session regarding administration of tests (124 teachers attended one of 6 offered sessions of approximately 30 minutes in length.) - Review test manual and other materials in preparation for administering tests. - Talk to principals and colleagues regarding scheduling changes and arrangements for LD, ESL., and absent students. - Count out testing materials and obtain additional items if needed. - Keep track of students needing makeup tests. - Check for accuracy of student name and identifying information on answer sheets. - Administer tests (3-5 days, 20 - 114 minutes per day depending on grade level) - Review answer sheets for unacceptable markings. - Organize tests for pick up. - Review test results. - Answer parent questions regarding test results. - File student results in individual student folders.

4. Secretarial staff:

Central office support staff: In addition to providing secretarial support to the workings of the testing committee, Central Office secretarial staff provided much of the clerical work for the 1989 achievement and ability testing. Major tasks included supplying and "bubbling" student identification numbers on answer sheets; sorting, counting out, and packaging testing materials by teacher or middle school teams; and preparing packets for the teacher inservice. These packets, which are intended to be reusable, contained both general information (e.g., how to describe the tests to students) and distinct information for each grade level. Each packet also included an examiner's manual for both CAT and TSC. copy of student tests, practice tests, extra materials for the TSC memory test 389

(to be studied by teachers ahead of time), and a header sheet on which to record the number tested by each teacher.

Time estimates for these major tasks include:

- Preparing teacher inservice packets: 10 hours - Preparing testing materials for each school: 16 hours - "Bubbling" student identification numbers on answer sheets: 7 days (56 hours) - Scanning, editing and Bcoring answer sheets and preparing and sorting parent, teacher and building reports: more than full-time for 3 weeks

These estimates do not include time spent identifying and ordering needed materials; checking in ordered materials as they arrived from the publisher*, organizing and typing information for teachers' pa’ckets; sorting, counting, packaging and labeling reusable testing materials to for use next year.

School secretaries: Each elementary and middle school building was responsible for notifying parents about the testing two weeks ahead of time and placing 1800 parent report forms in envelopes addressed to each home. These tasks usually fell to building secretaries. In addition, secretaries helped collect testing materials from teachers in preparation for delivery to the central office.

5. Students: (approximately 1800) 3-5 days, 20-114 minutes per day depending on grade level: Grade 3: total testing time: 219 minutes 4: 4 33 6: 433 8: 398

Teachers typically describe students as "drained" after testing so other instructional activities for the week were planned with student fatigue in mind. LIST OF REFERENCES

Abravanel, H. (1983). Mediatory myths in the service of organizational ideology. In L. R. Pondy, P, J. Frost, G. Morgan, T. C. Dandridge (Eds.), Organizational symbolism (pp. 273-293), Greenwich, CT: JAI Press.

Airasian, P. W. (1987). State mandated testing and educational reform: Context and consequences. American Journal of Education. 95(3). 393-412.

Airasian, P. W. (1988). Symbolic validation: The case of state-mandated, high stakes testing. Educational Evaluation and Policy Analysis. 10(4) . 301-313.

Anderson, G. L. (1987, April). Toward a critical ethnography of educational administration. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC.

Anderson, G. L. (1989). Critical ethnography in education: Origins, current status, and new directions. Review of Educational Research. 59 (3), 249-270.

Anderson, G. L. (1990). Toward a critical constructivist approach to school administration: Invisibility, legitimation, and the study of non-events. Educational Administration Quarterly. 26(1). 38-59.

Anderson, G. L. , & Kelley, D, M. (1987, April). Negotiating organizational reality: The mutual validation of research outcomes. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC.

Anyon, J. (1981). Social c Ibsb and school knowledge. Curriculum Inquiry. 11 (1), 3-42.

Argyris, C., Putnam, R., McLain, D. (1985). Action Science. San Francisco: Jossey-Bass.

390 391

Argyris, C. & Schon, D. A. (1978). Organizational learning: A theory of action perspective. Reading, MA: AddiBon-Wesley.

Ball, S. P. (1984). Beachside reconsidered: Reflections on a methodological apprenticeship. In R. G. BurgeBS (Ed.), The research process in educational settings: Ten case studies (pp. 69-96). London and New York: Falmer Press.

Bardach, E. (1977). The implementation game. Cambridge, MA: MIT Press.

Baron, J. B. (1989). Performance testing in Connecticut: Educational Leadership 46(7), 8.

Bateson, G. (1972). Steps to an ecology of the mind. New York: Ballentine Books.

Bateson, G. (1979). Mind and nature. New York: E. P. Dutton.

Belenky, M. F., Clinchy, B. M . , Goldberger, N. R . , Tarule, J . M . (1986). Women’s ways of knowing: The development of self, voice, and mind. New York: Basic Books.

Bennis, W., & Nanus, B, (1985). Leaders: The strategies for taking charge. New York: Harper & Row.

Berger P. L., & Luckmann, T. (1966). The social construction of reality: A treatise in the sociology of knowledge. New York: Anchor Books.

Berman, P., & McLaughlin, M. W. (1977). Federal Programs supporting educational change. Vol. VII: Factors affecting implementation and continuation. Santa Monica, CA: Rand Corporation.

Berman, P., & McLaughlin, M. W. (1978). Federal programs supporting educational change. Vol. VIII: Implementing and sustaining innovations. Santa Monica, CA: Rand Corporation.

Bernstein, B. (1971). On the classification and framing of educational knowledge. In M. F. D. Young (Ed). Knowledge and control: New directions for the sociology of education. London: Collier Macmillan.

Blumberg, A., 4 Greenfield, W. (1986). The effective principal: Perspectives on school leadership. 2nd ed. Boston: Allyn and Bacon. 392

Blumer, H. (1969). Symbolic interactionism: Perspective and method. Berkeley, CA: The University of California Press.

Bogdan, R. C. , & Biklen, S. K. (1982). Qualitative research for education: An introduction to theory and method. Boston: Allyn and Bacon,

Bormann, E. G. (1983). Symbolic convergence: Organizational communication and culture. In L. L. Putman & M. E. Pacanowsky (Eds.), Communication and organizations: An interpretive approach (pp. 92-123). Beverly Hills, CA: Sage.

Boyd, W, L. (1987). Public education’s last hurrah: Schizophrenia, amnesia, and ignorance in school politics. Educational Evaluation and Policy Analysis. 9(2), 85-100.

-Burrell, G., & Morgan, G. (1979). Sociological paradigms and organisational analysis: Elements of the sociology of corporate life. Heinemann, London.

California Assessment Project Staff (1989). Authentic Assessment in California. Educational Leadership. 46(7), 6.

Carr, W. , &. Kemmis, S. (1986). Becoming critical: Education, knowledge and action research. London and Philadelphia: Falmer Press.

Carroll, J. B. (Ed.). (1956). Language, thought, and reality: Selected writings of Benjamin Lee Whorf. Cambridge, MA: MIT Press.

Chance, W. (1988). ...the Best of Educations. Denver, CO: Education Commission of the States.

Cicourel, A. V. (1973). Cognitive psychology. Harmondsworth: Penguin.

Clune, W. H. (1987). Institutional choice aB a theoretical framework for research on educational policy. Educational Evaluation and Policy Analysis. 9 . 117-132.

Cohen, D. K. (1987). Educational technology, policy and practice. Educational Evaluation and Policy Analysis. 9, 153-170. 393

Conley, S. C. (1988). Reforming paper pushers and avoiding free agents: The teacher as constrained decision maker. Educational Administration Quarterly. 24(4). 393-404.

Corbett, H. D. , Firestone, W. A. , & Rossman, G. (1987). Resistance to planned change and the sacred in school cultures. Educational Administration Quarterly. 23(4). 36-59.

Cusick, P. A. (1973). Inside high school: The student*s world. New York: Holt, Rinehart, & Winston.

Darling-Hammond, L. (1988). The futures of teaching. Educational Leadership. 46(3). 4-10.

Darling-Hammond, L., & Berry, B. (1988). The evolution of teacher policy. Santa Monica, CA: Rand Corporation.

Deal, T., & Wiske, M. S. (1983). Planning, plotting, and playing in education’s era of decline. In J. V. Baldridge & T. Deal (Eds.), The dynamics of organizational change in education, (pp. 451-472), Berkeley, CA: McCutchan.

Donmoyer, R. (1985). Cognitive anthropology and research on effective principals. Educational Administration Quarterly. 21. 31-57.

Donmoyer, R. (1988, June). Generalizability and the single case study. A paper presented at the Stanford University Conference on Qualitative Research Issues, Palo Alto, CA.

Deetz, S. A., & Kersten, A. (1983). Critical models of interpretive research. In L. L. Putman & M. E. Pacanowsky (Eds.), Communication and organizations: An interpretive approach (pp. 147-171). Beverly Hills, CA: Sage.

Dubnick, M. J., & Bardis, B. A, (1983). Thinking about public policy: A problem solving approach. New York: John Wiley.

Dunn, W. N. (1981). Public policy analysis: An introduction. Englewood Cliffs, New Jersey: Prentice-Hall.

Duranti, A. (1988). Ethnography of speaking: Toward a linguistics of praxis. In F. J. Newmeyer (Ed.), Language the socio-cultural context (pp. 210-228) Vol IV. Linguistics: The Cambridge survey. Cambridge: Cambridge University Press. 394

Edelsky, C. (1978). Genderlects: A brief review of the literature. Paper presented at annual meeting of NCTE, Kansas City. (ERIC Document Reproduction Service No. ED 165 187)

Edelsky, C. (1981). Who's got the floor? Language in society. 10. 383-421.

Elmore, R. (1977). Lessons from follow through. Policy Analysis. 1. 549-584.

Elmore, R. F. (1983), Complexity and control: What legislators and administrators can do about implementing public policy. In L. Shulman & G. Sykes, (Eds.) Handbook of Teaching and Policy (pp. 342-369). New York: Longman.

Elmore, R. (1987). Reform and the culture of authority in schools. Educational Administration Quarterly. 23.(4). 60-78.

Elmore, R. F., & McLaughlin, M. W, (1982). Strategic choice in federal education policy: The compliance - assistance trade-off. In A. Lieberman & M. W. McLaughlin (Eds.), Policy Making in Education. 81st Yearbook of the National Society for the Study of Education (pp. 159-194). Chicago, IL: University of Chicago Press.

Elmore, R. F., & McLaughlin, M. W. (1983). The federal role in education: Learning from experience. Education and Urban Society. 15.(3). 309-330.

Ellwein, M. C., Glass, G. V., & Smith, L. E. (1988). Standards of competence: Propositions on the nature of testing reforms. Educational Researcher. 17. (8). 4-9.

Erickson, F. (1986). Qualitative methods in research on teaching. In M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed.) (pp.119-161). New York: Macmillan.

Erickson, F. (1987). Conceptions of school culture: An overview. Educational Administration Quarterly. 23 (4), 11-24.

Evered, R. (1983). The language of organizations: The case of the Navy. In L. R. Pondy, P. J. Frost, G. Morgan, T. C. Dandridge (Eds.), Organizational symbolism (pp. 125-143). Greenwich, CT: JAI Press. 395

Fay, B. (1976). Social theory and political practice. New York: Holmes &. Meier.

Fay, B. (1977). How people change themselves: The relationship between critical theory and its audience. In T. Ball (Ed.), Theory and praxis (pp. 200-233). Minneapolis, MN: University of Minnesota Press.

Fay, B. (1987). Critical social science: Liberation and its limits. Ithica, NY: Cornell University Press.

Fetterman, D. M. (1989). Ethnography: Step by step. Beverly Hills: Sage.

Firestone, W. , & Bader, B. D. (1989, March). District coalitions for reform: A preliminary report on the application of coalition theory to district operations. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.

Foster, W. (1989). [Review of Ways and meanings of research in educational administration!. Educational Administration Quarterly. 25(1), 111-113.

Friere, P. (1973). Pedagogy of the Oppressed. New York: Seabury

Geertz, C. (1973). The interpretation of cultures. New York: Basic Books.

Galloway, C. M. (1977). Nonverbal: Authentic or artificial? Theory into Practice. 16. 129-133.

Galloway, C. M. (1984). Nonverbal and teacher-student relationships: An intercultural perspective. In A. Wolfgang (Ed.), Nonverbal behavior (411-430). Lewiston, NY: C. J. Hogrese.

Garfinkel, H. (1967). Studies in ethnomethodolbgy. Englewood Cliffs, NJ: Prentice-Hall.

Ginsberg, R . , & Berry, B. (1990). Experiencing school reform: The view from South Carolina. Phi Delta Kappan, H(7), 549-552.

Gitlin, A., Siegel, M . , & Boru, K. (1988, April). Purpose and method: Rethinking the use of ethnography by the educational left. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. 396

Goffman, E. (1971). Relations in public. New York: Basic Books.

Goffman, E. (1974). Frame analysis. Boston: Northeastern University Press.

Goffman, E. (1983, February). The interaction order. American Sociological Association, 1982 Presidential Address. American Sociological Review. 48. 1-17).

Goodenough, W. (1981). Culture, language, and society. Menlo Park, CA: Benjamin-Cummings.

Goodlad, J. (1984), A place called school. New York: McGraw-Hill.

Goodlad, J. I, (1988-89). Understanding schools is basic to improving them. National Forum of Applied Educational Research Journal. 1(2). 1-9.

Green, J. L., & Harker, J. 0. (1988). Multiple perspectives analysis of classroom discourse. Norwood, N J : Ablex.

Green, J. L. , &. Weade, R. ( 1985). Reading between the words: Social cues for lesson participation. Theory into Practice. 24. 14-21.

Greenfield, T. (1980). The man who comes back through the door in the wall: Discovering self, discovering organizations. Educational Administration Quarterly. 16.(3), 26-59.

Greenfield, T. (1983, April). Environment as subjective reality. Paper presented the annual meeting of the American Educational Research Association,.Montreal.

Greenfield, T. (1985). Theories of educational organization: A critical perspective. International Encyclopedia of Education: Research and Studies. Ontario: Ontario Institute for Studies in Education.

Gusfield, J. (1976). The literary rhetoric of science: Comedy an pathos in drinking driver research. American Sociological Review. 41. 6-34,

Gusfield, J. (1979). "Buddy, can you paradigm?": The crises of theory in the welfare state. Pacific Sociological Review. 22(1). 3-27. 397

Hall, E. T. (1974). Handbook for proxemic research. Washington, DC: Society for the Anthropology of Visual Communication.

Hall, E. T. (1976). Beyond culture. Garden City, NY: Anchor Books.

Haney, W., & Madaus, G. (1989). Searching for alternatives to standardized tests: Whys, whats, and withers. Phi Delta Kappan. 70(9). 683-687.

Hawley, W. D. (1988). Missing pieces of the educational reform agenda: Or, why the first and second waves may miss the boat. Educational Administration Quarterly. M<4), 416-437.

Healy, P. (1986). Interpretive policy inquiry: A response to the limitations of the received view. Policy Sciences. 19(4). 381-396.

Heron, J. (1981). Philosophical basis for a new paradigm. In P. Reason & J. Rowan (Eds.), Human inquiry: A sourcebook of new paradigm research.

Heron, J. (1988). Validity in cooperative inquiry. In P. Reason (Ed.), Human inquiry in action: Developments in new paradigm research (pp. 41-59). Beverly Hills, CA: Sage.

Hymes, D. (1962). The ethnography of speaking. In T. Gladwin & W. C. Sturtevant (Eds.), Anthropology and human behavior (pp. 13-53). Washington, DC: Anthropological Society of Washington.

Johnston, A. P., &. Niedermeier, H. G. (1987). State sponsored education reform in 1980*b : Schools as policy .junkyards. (ERIC Document Reproduction Service, No. EJ 371 956)

Kerr, D. (1976). Educational policy: Analysis. Structure, and Justification. NY: David McKay.

Kirst, M. W. (1988). Recent state education reform in the United States: Looking forward and backward. Educational Administration Quarterly. 24 (3). 319-328.

Krim, R. (1988). Managing to learn: Action inquiry in city hall. In P. Reason (Ed.), Human inquiry in action: Developments in new paradigm research (pp. 144-162). Beverly Hills, CA: Sage. 398

Kroeber, A. L. , & Parsons, T. (1958). The concepts of culture and of social systems. American Sociological Review, 23. 582-583.

Lakoff, G., & Turner, M. (1989). More than cool reason: A field guide to poetic metaphor. Chicago: University of Chicago Press.

Lasswell, H. D. (1977). A pre-view of policy sciences. New York: American Elsevier.

Lather, P. (1986). Research as praxis. Harvard Educational Review. 56(3). 257-277,

Lather, P. (1988, October). Advocacy, methodology, and reflexivity: Postmodernism, praxis and social change. Paper presented at the tenth annual conference on Curriculum Theory and Classroom Practice, Dayton, OH.

Lewis, M. & Simon, R, (1986). A discourse not intended for her: Learning and teaching within patriarchy. Harvard Educational Review 56(4). 457-472.

Lincoln, Y. S., &. Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.

Livingston, C., Castle, S., Nations, J. (1989). Testing and curriculum reform: One school's experience. Educational Leadership. 46.(7). 23-25,

Louis, M. R. (1983). Organizations as culture-bearing milieux. In L. R. Pondy, P. J. Frost, G. Morgan, T. C. Dandridge (Eds.), Organizational symbolism (pp. 39-54). Greenwich, CT: JAI Press,

Lowi, T. (1972). Four systems of policy, politics and choice. Public Administration Review. July-August, 298-310.

Madaus, G. F. (1985). Test scores as administrative mechanisms in educational policy. Phi Delta Kannan. 66.(9). 611-617.

Maguire, P. (1987). Doing participatory research: A feminist approach. Amherst, MA: University of Massachusetts.

Mandelbaum, D. G. (Ed.). Selected writings of Edward Sapir in language, culture and -personality. Berkeley, CA: University of California Press. 399

McDonnell, L. M. , & Elmore, R. F. (1987). Getting the job done: Alternative policy instruments. Educational Evaluation and Policy Analysis. 9(2). 133-152.

McLaughlin, M. W. (1976). Implementation as mutual adaptation: Change in classroom organization. Teachers College Press. 77(3) . 339-351.

McLaughlin, M. W. (1987). Learning from experience: Lessons from policy implementation. Educational Evaluation and Policy Analysis. 9 (2). 171-178.

McNeil, L. M. (1988a). Contradictions of Control Part 1: Administrators and teachers. Phi Delta Kappan. 69(5). 333-339.

McNeil, L. M. (1988b). Contradictions of Control Part 2: Teachers, students, and curriculum. Phi Delta Kappan 69(6), 432-438.

McNeil, L. M. (1988c). Contradictions of Control Part 3: Contradictions of reform. Phi Delta Kappan 69(7). 478-485.

Metz, M. H. (1984, April). Ethnography, organizational theory, and educational innovation. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Miller, L., & Lieberman, A. (1988). School improvement in the United States: Nuance and numbers. Qualitative Studies in Education. 1(1). 3-19.

Mishler, E. G. (1986). Research interviewing: Context and narrative. Cambridge, MA: Harvard University Press.

Mitchell, D. E. (1984). Educational policy analysis: The state of the art. Educational Administration Quarterly. 20(3), 129-160.

Mitchell, D. E., & Encarnation, D. J. (1984). Alternative state policy mechanisms for influencing school performance. Educational Researcher. 13(5). 4-11.

Mooney, Ross (1975). The researcher himself. In W. Pinar (Ed.) Curriculum theorizing: The reconceptualists (pp. 173-207). Berkeley, CA: McCutcheon Publishing.

Morgan, G. (Ed.), Beyond method: Strategies for social research. Beverly Hills: Sage. 400

Morgan, G. (1986). Images of Organizations. Beverly Hills: Sage.

Morgan, G., Frost, P. J., & Pondy, L. R. (1983). Organizational symbolism. In L. R. Pondy, P. J. Frost, G. Morgan, T. C. Dandridge (Eds.), Organizational symbolism (pp. 3-35). Greenwich, CT: JAI Press.

Mumby, D. K. (1988). Communication and power in organizations: Discourse, ideology, and domination. Norwood, NJ: Ablex.

Murphy, J. T. (1980). The state role in education: Past research and future directions. Educational Evaluation and Policy Analysis. 2 (3). 39-51.

Neill, D. M., & Medina, N. J. (1989). Standardized testing: Harmful to educational health. Phi Delta Kappan. 70(9). 688-697.

Nickerson, R. S. (1988). New directions in educational assessment. Educational Researcher. 18.(9). 3-7.

Odden, A., &. Marsh, D. (1988). How comprehensive reform legislation can improve secondary schools. Phi Delta Kappan» 69(8). 593-598.

Ohio Department of Education (1987, July), Ohio’s statewide testing program: Assuring student competency. Columbus, OH: Author.

Ohio Department of Education. (1989, Spring). Teacher Update. 3 . 2. Columbus, OH: Author.

Orlich, D. C. (1985). The dilemma of strong traditions and weak empiricism. Teacher Education Quarterly. 12. 23-32.

Orlich, D. C. (1989). Education reforms; Mistakes, misconceptions, miscues. Phi Delta Kappan. 70(7). 512-517.

Passow, A. H. (1988). Whither (or wither?) school reform? Educational Administration Quarterly. 24(3). 246-256.

Pearson, D. P., & Valencia, S. (1987). Assessment, accountability, and professional prerogative In J. A. Niles & R. V. Lalik (Eds.), Research in literacy: Merging perspectives. 36th Yearbook of the National Reading Conference. Rochester, NY: National Reading Conference. 401

Peshkin, A. (1988). In search of subjectivity: One's own. Educational Researcher. 17(7). 17-22.

Philips, W. W. (1985). Participant structure and communicative competence: Warm Springs children in community and classroom. In C. B. Cazden, V. P. John, & D. Hymes, (Eds.), Functions of language in the classroom (pp. 370-394). Prospect Heights, IL: Waveland Press.

Pipho, C. (1986). States move reform closer to reality. Phi Delta Kappan, 68(4). K1-K8-.

Pondy, L. R. (1983). The role of metaphors and mythsin organizations and in the facilitation of change. In L. R. Pondy, P. J. Frost, G. Morgan, T. C. Dandridge (Eds.), Organizational symbolism (pp. 157-166). Greenwich, CT: JAI Press.

Popham, W. J. (1987). The merits of measurement-driven instruction. Phi Delta Kappan. 68(9). 679-682.

Powell, A. G., Farrar, E, Cohen, D. K. (1985). The shopping mall high school: Winners and losers in the educational marketplace. Boston: Houghton-Mifflin.

Pressman, J. L., & Wildavsky, A. (1984). Implementation. 3rd ed. Berkeley: University of California Press.

Pugh, W. C. (1987, March). Ethnography and school improvement research: Analyzing metaphoric language in the interpretation of instructional leadership. Paper presented at the annual forum on Ethnography in Education Research, Philadelphia, PA.

Prunty, J. J. (1984). A critical reformulation, of educational policy analysis. Victoria: Deakin University.

Purkey, S. C., & Smith, M. S. (1983). School reform: The policy implications of the effective schools literature. Paper prepared for the National Institute of Education Conference on State and Local Policy Implications of Effective Schools Research. (ERIC Document Reproduction Service No. ED 245 350)

Reason, P. (Ed.). (1988). Human inquiry in action: Developments in n^w paradigm research. Beverly Hills: Sage. 402

Reason, P., & Hawkins, P. <1988). Storytelling as inquiry. In P. Reason (Ed.), Human inquiry in action: Developments in new paradigm research (pp. 79-101). Beverly Hills: Sage.

Reason, P., & Rowan, J, <1981a). Foreword. In P. Reason & J. Rowan (Ed.), Human inquiry: A sourcebook of new paradigm research, pp. xi-xxiv. New York: John Wiley & Sons.

Reason, P., & Rowan, J. (1981b). Issues of validity in new paradigm research. In P. Reason & J. Rowan (Ed.), Human inquiry: A Bourcebook of new paradigm research, pp. 239-250. New York: John Wiley & Sons.

Rein, M. & Schon, D. A. (1977). Problem setting in policy research. In C. H. Weiss (Ed.), Using social research in public policy making (pp. 235-251). Lexington, MA: D. C. Heath & Company.

Rein, M. , & White, S. H. (1977). Policy research: Belief and doubt. Policy Analysis 3.(2). 239-271.

Rittel, H., & Webber, M. (1973). Dilemmas in a general theory of planning. Policy Sciences. 4, 155-169.

Rosenstock-Huessy, E. (1970). Speech and reality. Norwich, VT: Argo Books.

Sabatier, P., & Mazmanian, D. (1980). The implementation of public policy: A framework of analysis. Policy Studies Journal, 8 (4), 538-560.

Sarason, S. B. (1971). The culture of school and the problem of change. Boston: Allyn and Bacon.

Schon, D. (1979). Generative metaphor: A perspective on problem setting in social policy. In A. Ortony, (Ed). Metaphor and thought (pp. 254-283). Cambridge: Cambridge University Press.

Schwartz, P., & Ogilvy, J. (1979). The emergent paradigm: Changing patterns of thought and belief. Analytical Report 7, Values and Lifestyles Program. Menlo Park, CA: SRI International.

Scott, W. R. (1981). Developments in organizational theory, 1960-1980. American Behavioral Scientist, 24(3). 407-422. 403

Sergiovanni, T. , & Corbally, S. (Eds.), Leadership and organisational culture. Champaign, 1L: University of Illinois Press.

Shepard, L. (1989). Why we need better assessments. Educational Leadership. 46(7). 4-7.

Shujaa, M. (1989, February). Who teaches to the test? Using multisite qualitative research to examine the Influence of state and local testing policies on teachers' perceptions and behavior. Paper presented at the Tenth Annual University of Pennsylvania Ethnography in Education Research Forum, Philadelphia, PA.

Shulman, L. S. (1983). Autonomy and obligation: The remote control of teaching. In L. S, Shulman G. Sykes (Eds.), Handbook of teaching and policy (pp. 484-505). New York: Longman.

Simon, R. I., & Dippo, D. (1986). On critical ethnographic work. Anthropology and Education Quarterly. 17(4). 195-202.

Smircich, L. & Morgan, G. (1982). Leadership: The management of meaning. Journal of Applied Behavioral Science. 18(3), 257-273.

Smith, H. K. (1988). The evaluator/researcher as person vs. the person as evaluator/researcher. Educational Researcher. 17(2). 18-23.

Soltis, J. F. (1988). Reform or reformation? Educational Administration Quarterly. 24(3). 241-245.

Stacey, J. (1988). Can there be a feminist ethnography? Women's Studies International Forum. 11(1L. 21-27. I Strauss, A. L, (1987). Qualitative analysis for social scientists. Cambridge: Cambridge University Press.

Taylor, B. O. (1986). How the effective elementary principal accomplishes school improvement. Paper presented at annual meeting of the American Educational Research Association, San Francisco, CA.

Thayer, L* (1988). Introduction to the series: People, communication, and organization. In Mumby, D. K. Communication and power in organizations: Discourse, ideology and domination (ix-xii). Norwood, NJ : Ablex. 404

Theodossin, E. (1982). The management, of self-interest: Phenomenology and staff motivation. Coombe Lodge Working Paper: Information Bank No. 1634. (ERIC Document Reproduction Service No.. ED 252 753).

Timar, T. B. , & Kirp, D. L. (1988). Managing educational excellence. New York: Stanford Series on Education and Public Policy, Falmer Press.

Timar, T. B. , & Kirp, D. L. (1989). Education reform in the 1980’s: Lessons from the states. Phi Delta Kappan. 70(7), 504-511.

Torbert, W. R. (1981). Why educational research has been so uneducational: the case for a new model of social science based on collaborative inquiry. In P. Reason & J. Rowan (Ed.), Human inquiry: A sourcebook of new paradigm research, pp. 141-151. New York: John Wiley & Sons.

Trughill, P. (1983). Sociolinguistics: An introduction to language and society. Harmondsworth: Penguin. van Dijk, T. A. (1985). Introduction: Dialogue as discourse and interaction. In T. A. van Dijk (Ed.), Discourse and dialogue (pp. 1-11). Vol. 3 of Handbook of discourse analysis. London: Academic Press.

Van Maanen, J. (1988). Tales of the field: On writing ethnography. Chicago: University of Chicago Press. van Manen, M. (1986). Bibliography. Human Science in Education Project. University of Alberta, Edmonton.

Van Meter, D. S., & Van Horn, C. E. (1975). The policy implementation process, a conceptual framework. Administration and Society. 6 (4). 445-448.

Wax, R. (1971). Doing fieldwork: Warnings and advice. Chicago: University of Chicago Press.

Weatherley, R. , & Lipsky, M. (1977), Street level bureaucrats and institutional information: Implementing special education reform. Harvard Educational Review. 47. 171-197.

Weick, K. E. (1979). The social psychology of organizing. Reading, MA: Addison-Wesley. 405

Weiss, C. H. (1982). Policy research in the context of diffuse decision-making. Journal of Higher Education. 51(6), 620-639.

White, S. H. (1979). Children in perspective: Introduction to the section. American Psychologist. 34. 812-814.

Wiggins, G. (1989). Teaching to the (authentic) test. Educational Leadership. 46(7). 41-47.

Wilden, A. (1987). The rules are no game: The strategy of communication. London and New York: Routledge & Kegan Paul.

Wimpelberg, R. K., Teddlie, C., & Stringfield, S. (1989). Sensitivity to context: The past and future of effective schools research. Educational Administration Quarterly. 25fl). 82-107.

Wirt, F. M . , & Christovich, L. (1989). Administrators' perceptions of policy influence: conflict management styles and roles. Educational administration Quarterly. 25(1), 5-36.

Wise, A. (1979). Legislated learning. Berkeley, CA: University of California Press.

Wise, A. E., & Darling-Hammond, L. (1983). Beyond standardization: State standards and school improvement. Paper prepared for the National Institute of Education Conference on State and Local Policy Implications of Effective Schools Research. (ERIC Document Reproduction Service No. ED 245 353)

Wolcott, H. S. (1973). The man in the principal's office: An ethnography. New York: Holt, Rinehart & Winston.

Yin, R. K. (1984). Case study research: Design and methods. Beverly Hills, CA: Sage.