Kasprzyk, Daniel the Schools and Staffing Survey. Recommendations
DOCUMENT RESUME
ED 404 370 TM 026 456
AUTHOR Mullens, John E.; Kasprzyk, Daniel TITLE The Schools and Staffing Survey. Recommendations for the Future. Conference Proceedings. INSTITUTION National Center for Education Statistics (ED), Washington, DC.; Policy Studies Associates, Inc., Washington, DC. REPORT NO ISBN-0-16-048961-X; NCES-97-596 PUB DATE Dec 96 NOTE 248p. AVAILABLE FROM U.S. Government Printing Office, Superintendent of Documents, Mail Stop: SSOP, Washington, DC 20402-9328. PUB TYPE Collected Works Conference Proceedings (021)
EDRS PRICE MF01/PC10 Plus Postage. DESCRIPTORS Databases; Data Collection; Educational Change; Educational Finance; *Educational Policy; Elementary Secondary Education; *Futures (of Society); *National Surveys; Private Schools; Public Schools; *Research Utilization; Teacher Characteristics; *Teacher Supply and Demand IDENTIFIERS *Schools and Staffing Survey (NCES)
ABSTRACT The Schools and Staffing Survey (SASS) was designed by the National Center for Education Statistics (NCES) to provide recurrent information about public and private elementary and secondary schools, teachers, and administrators. The NCES commissioned these papers to examine the SASS and to make recommendations about its future. The following papers, presented at 1996 NCES seminars, are included:(1) "Should SASS Measure Instructional Processes and Teacher Effectiveness?" (Susan Stodolsky);(2) "Toward an Organizational Database on America's Schools: A Proposal for the Future of SASS, with Comments on School Reform, Governance, and Finance" (David P. Baker);(3) "Technology for K-12 Education: Asking the Right Questions" (Kathleen Fulton); (4) "Linking Student Data to SASS: Why, When, How" (Phillip Kaufman); (5) "Making Data Relevant for Policy Discussions: Recommendations for Redesigning the School Administrator Questionnaire for the 1998-99 SASS" (Henry Zheng); (6) "Measures of Inservice Professional Development: Suggested Items for the 1998-99 Schools and Staffing Survey" (Dorothy M. Gilford);(7) "District-Level Data in the Schools and Staffing Survey" (J. Michael Ross);(8) "Use of Education Information Systems with the Schools and Staffing Survey: How Can SASS Be Linked to Schools?" (Rolf K. Blank);(9) "Collecting Representative Data on School Resources: Understanding the Linkage between Adequacy, Equity, and Opportunity To Learn through SASS" (Jay G. Chambers); (10) "The Schools and Staffing Survey for 1998-99: Design Recommendations To Inform Broad Education Policy" (Erling E. Boe);(11) "1998-99 Schools and Staffing Survey: Issues Related to Survey Depth" (Susan P. Choy); and (12) "Reflections on the Papers Prepared for the Schools and Staffing Survey Seminar Series" (John Howard Burkett). Each paper contains references. (SLD) TM
A I A I A
I I 1
I
EPARTMENT OF EDUCATION U S Improvement Officef Educational Research and EDU ATIONAL RESOURCESINFORMATION CENTER (ERIC) This document has beenreproduced as received from the person ororganization originating it CI Minor changes have beenmade to improve reproduction quality
Points of view or opinions stated inthis document do not necessarilyrepresent official OERI position or policy
' - O ST COPY AVAILABLE 2 NATIONAL CENTER FOR EDUCATION STATISTICS
Conference Proceedings December 1996
The Schools and Staffing Survey Recommendations for the Future
John E. Mullens Policy Studies Associates, Inc.
Daniel Kasprzyk National Center for Education Statistics
U.S. Department of Education Office of Educational Research and Improvement NCES 97-596 U.S. Department of Education Richard W. Riley Secretary Office of Educational Research and Improvement Sharon P. Robinson Assistant Secretary National Center for Education Statistics Pascal D. Forgione, Jr. Commissioner
The National Center for Education Statistics (NCES) is the primary federal entity for collecting, analyzing, and reporting data related to education in the United States and other nations. It fulfills a congressional mandate to collect, collate, analyze, and report full and complete statistics on the condition of education in the United States; conduct and publish reports and specialized analyses of the meaning and significance of such statistics; assist state and local education agencies in improving their statistical systems; and review and report on education activities in foreign countries. NCES activities are designed to address high priority education data needs; provide consistent, reliable, complete, and accurate indicators of education status and trends; and report timely, useful, and high quality data to the U.S. Department of Education, the Congress, the states, other education policymakers, practitioners, data users, and the general public. We strive to make our products available in a .variety of formats and in language that is appropriate to a variety of audiences. You, as our customer, are the best judge of our success in communicating information effectively. If you have any comments or suggestions about this or any other NCES product or report, we would like to hear from you. Please direct your comments to: National Center for Education Statistics Office of Educational Research and Improvement U.S. Department of Education 555 New Jersey Avenue NW Washington, DC 20208-5574
December 1996
The NCES World Wide Web Home Page is http://www.ed.gov/NCES/
Suggested Citation U.S. Department of Education. National Center for Education Statistics. The Schools and Staffing Survey: Recommendations for the Future, NCES 97-596, by John E. Mullens and Dan Kasprzyk. Project officer, Mary Rollefson. Washington, DC: 1996.
Contact: Mary Rollefson (202) 219-1336
For sale by the U.S. Government Printing Office Superintendent of Documents, Mail Stop: SSOP, Washington, DC 20402-9328 ISBN 0-16-048961-X PREFACE
The Schools and Staffing Survey (SASS) was first administered in 1987-88 by the National Center for Education Statistics (NCES) to provide recurrent information on public and private elementary and secondary schools, teachers, and administrators, especially data on conditions affecting supply and demand for teachers and the characteristics of the teacher force. By linking survey data from classroom teachers and individual school sites to information from local education agencies (LEAs), the survey provides recurrent information on public and private elementary and secondary schools, teachers, and administrators to inform state and federal decisions, and the educational research community. The data now collected fall into four general categories:
Critical components of teacher supply, demand, and attrition, with attention to critical shortage areas and the policies and practices at all levels enacted to meet the demand in those areas
The professional characteristics, preparation, and experience of teachers and administrators, plus their perceptions of school conditions, professional responsibilities, decision making, and compensation policies
The Conditions and characteristics of the school as a work place and a learning place, including characteristics of the student body, curriculum, special programs, and organizational structure
The implementation of school programs and policies such as English as a second language, bilingual education, diagnostic and prescriptive services, and programs for the gifted and talented
The current survey design and process provide for a network of interlocking datasets from different organizational elements at the classroom, school, and district levels. The target population includes all elementary and secondary schools, teachers, and principals in the U.S. The process and instruments are as follows:
1. The School Administrator Survey and the School Survey are administered to a sample of 9,784 public and 3,360 private schools
2. The Teacher Demand and Shortage Survey is administered to each sampled private school (embedded in the school questionnaire) and to the 5,500 LEAs of the sampled public schools
3. The Teacher Survey is administered to a random sample of 56,736 public and 11,548 private school teachers in the sampled schools
5 This process results in a comprehensive, linkeddatabase that provides national estimates for public and private schools, districts, principals, and teachers;state-level estimates for public data and affiliation-specific estimates for private data.
SASS was administered at three-year intervals from 1987-88through 1993-94, with a five year interval before its next administration in 1998-99. NCES is examining thedirection, purposes, and uses for SASS in the twenty-first century. This includesscrutinizing the current uses of its data, its relationships with other federally sponsored datacollection projects, and future nationalsurvey needs during a period of evolving policy priorities.
As part of this process, NCES commissioned twelvepapers to examine SASS and make recommendations about improving the scope and utility of thesurveys. Authors were selected from the ranks of experts working to understand and describe the nation'sschools, and policymakers interested in instructional practice and professional development.They represent academia, the research community, and specialists in technology, teacher education,and state and local data collection. Authors were asked to examine the current SASS, address theeffects of specific issues on future iterations of SASS, and make recommendations about improving,focusing, or expanding the scope and utility of the surveys. The papers were presented during seminars at NCES in theearly part of 1996.
The first paper, by Susan S. Stodolsky, addresses data collectionon instructional practices and teaching effectiveness. Current reform movements embodyan expectation that changes in teachers' instructional practices will play an important role in improving studentachievement, and that national measurement of those changes in classroom processes will track theprogress of reform.Instruction and teaching effectiveness are central to the educationalprocess and therefore, one might presume, also central to our collection of information about education. Stodolskypresents the argument for collecting data on instructional practices and teaching effectiveness; examineshow teaching effectiveness is conceptualized; describes strengths and limitations of observationaland survey data collection; suggests ways in which the effect of curricular reforms could beassessed; and proposes specific ways that SASS might measure instructional practice andcontent.
In the next paper, David P. Baker suggests that SASS becomea new organizational database for the nation's K through 12th grade schools:an omnibus survey about the internal organization of elementary and secondary schools. Baker proposes that the foremost priorityof SASS ought to be organizational and managerial information focusingon four main perspectives: school organization, multiple levels of governance, financial resources and flows, and school-leveleducational outcomes. Baker argues that if SASS were to becomea central vehicle for NCES, it is essential to collect
ii school-level data on educational outcomes such as student achievement, promotion, dropout, disciplinary actions, and college applications. SASS data could then contribute to the policy debate linking student outcomes with schools, their organizations, and resources.
Some analysts suggest that computer use will completely transform classroom instruction in the next 20 years.In the third paper, Kathleen Fulton says that drastic changes are necessary in our data collection about technology if it is to keep pace with classroom innovations. While current data collection on classroom use of computers and related equipment may respond to public and congressional interest in defining the scope of computer use, it is primarily limited to numbers and availability of computers. Future data collection, she argues, needs to be refocused and directed toward defining the effect of state policies on access to technology in schools, how computers are actually used by teachers and students, and the effect of that use on teaching and learning.
Phillip Kaufman argues that SASS data ought to link with information on student achievement. In his paper, Kaufman presents the feasibility and benefits of linking a student sample with SASS teacher and administrative data. He proposes that a successful merger of two data collection systems should produce data that could measure students' overall academic performance, their growth in achievement, and their progress through critical transitions. A linkage between SASS and a student data component, Kaufman suggests, should also produce some administrative or respondent efficiencies and analytical benefits.
Henry Y. Zheng's paper discusses the scope and uses of the SASS School Administrator Questionnaire. He argues that current and future efforts to understand and guide educational reform will increase the importance of administrator survey data, especially such information as demographic and educational data, and information on principals' attitudes toward school management issues such as the priorities of educational goals, seriousness of school problems, and thedistribution of decision- making power in schools. Zheng suggests ways in which NCES might encourage greater use of the resulting data and recommends questionnaire modifications to increase data relevance.
Dorothy M. Gilford's paper addresses data collection on teachers' inservice professional development. She proposes a framework with which to classify types of programs and discusses several current issues and their implications for professional development and data collection. Gilford recommends expanding the types of professional development items included in SASS; fielding a new computer coordinator survey; and eliminating the district survey by incorporating its essential questions into the principal survey. Gilford also notes that data collection must accommodate the current broad spectrum of professional development activities and their slow evolution from simple
iii 7 awareness programs designed to inform teachers about new ideas tomore complex systemic programs shaped by constructivist principles and directed towardresults-driven education.
J. Michael Ross argues that the SASS samplingprocess ought to be redesigned to directly sample districts rather than schools. Given the importance ofdistrict-level data for systematically assessing the increased changes, complexities, and responsibilitiesin the organizational structures of schools and districts, Ross suggests that districts be sampledfirst, and then schools within the selected districts, a reversal on the current sample design. Redesigneddistrict surveys should also de- emphasize teacher demand and supply issues to focuson district policy and reform information. Such changes, he suggests, would help NCES assemble importantinformation that will be critical in assessing school reform.
Rolf K. Blank recommends that the current design of SASS belinked with state and local education information systems to provide direct and importantdata on the characteristics of American schools and how education is carried out within them. Sucha linkage, Blank asserts, would add to the usability and relevance of SASS data and increase state-leveldata analysis and reporting. He details three possible approaches and suggests NCES consider providingincentives for cooperation in state and local data collection.
Jay G. Chambers also suggests a redesign of SASS to facilitateexamination of resource allocation patterns in public and private schools. Chambersargues that such information would be valuable to researchers and other datausers without unduly increasing respondent burden. Chambers' paper and recommendations focus on personnel data since 80 percent of public schooldistrict budgets are devoted to personnel costs. Implementing his suggestions, hesays, would provide a foundation for addressing issues of equity, adequacy, and opportunityto learn within school systems.
In a comprehensive examination of SASS, Er ling E. Boereconsiders the goals, foci, and strategy of SASS; the content balance, extent of coverage, redundancy ofcoverage, and potential new areas; and recommends data collection priorities. Boe recommends collecting dataon both "enduring" and "emerging" issues of policyconcern. He suggests continuing to collect data in ten areas fundamental to the education process; continuing to collect dataon the basic attributes of school principals, LEAs, and schools; and expanding data collection ineight areas of school governance/organization and instruction. Thepaper emphasizes public school data collection, and recommends that SASS data be made relevant to education policydevelopment at all levels, since the mix of federal, state and local influenceson schooling has been, and will continue to be, in flux.
iv Susan P. Choy examines the depth of SASS, focusing on the level at which estimates should ybe provided, the respondent pool, and the response burden. Choy suggests that the relevance and importance of the original survey purposes remain intact, and the survey is able to capture information on enduring issues, even though changing policy concerns of the early 1990s have shifted the focus of some questions. Choy suggests that SASS monitor the extent to which various types of proposed reforms are actually present in schools and classrooms, and collect more information to describe what goes on at the classroom level. Choy also recommends continuing to collect data with which to provide state- and private school affiliation-level estimates.
Finally, John Howard Burkett argues that there is a pressing public need for more state and local information on schools and that SASS must heed the public's need for data. He echoes Boe's call to focus on fundamental aspects of schooling that have been subject to major recent debates, policy action, or public concern. He suggests that the value of SASS will be realized only if it addresses education at the state and local levels.
Individually and collectively, these papers set an ambitious agenda for NCES and SASS and provide the basis on which the Center can make decisions on how best to focus or expand the future direction and emphasis of SASS.
In addition to the fine work of the authors, we also want to acknowledge the contributions of others that helped make this project a success. Within NCES, Sharon Bobbitt played a key role in conceptualizing the conference. When Sharon became Director of the Knowledge Applications Division in the Office of Educational Research and Improvement, Mary Rollefson ably guided the project to its conclusion. At Policy Studies Associates, John Mullens directed the project and was instrumental in bringing the conference and this publication to fruition. He was assisted by Eileen O'Brien, Janie Funkhouser, Amy Hightower, Ben Lagueruela, Kim Thomas, and Nancy Thornes. To each person, we extend our gratitude and appreciation.
Paul Planchon Dan Kasprzyk Surveys and Cooperative Systems Group Education Surveys Program National Center for Education Statistics National Center for Education Statistics TABLE OF CONTENTS
Page
Preface
Should SASS Measure Instructional Processes and Teacher Effectiveness? 1 Susan S. Stodolsky, University of Chicago
Toward an Organizational Database on America's Schools: A Proposal for the Future of SASS, with Comments on School Reform, Governance, and Finance 19 David P. Baker, The Catholic University of America and American Institutes for Research
Technology for K-12 Education: Asking the Right Questions 39 Kathleen Fulton, University of Maryland
Linking Student Data to SASS: Why, When, How 53 Phillip Kaufman, MPR Associates, Inc.
Making Data Relevant for Policy Discussions: Recommendations forRedesigning the School Administrator Questionnaire for the 1998-99 SASS 67 Henry Y. Zheng, Ohio State University
Measures of Inservice Professional Development: Suggested Items for the1998-99 Schools and Staffing Survey 93 Dorothy M Gilford, Bethesda, MD
District-level Data in the Schools and Staffing Survey 135 J. Michael Ross, National Center for Education Statistics
Use of Education Information Systems with the Schools and StaffingSurvey: How Can SASS be Linked to Schools? 159 Rolf K. Blank, Council of Chief State School Officers
Collecting Representative Data on School Resources: Understanding theLinkage between Adequacy, Equity, and Opportunity to Learn Through SASS 167 Jay G. Chambers, American Institutes for Research
The Schools and Staffing Survey for 1998-99: Design Recommendations to Inform Broad Education Policy 185 Erling E. Boe, University of Pennsylvania
1998-99 Schools and Staffing Survey: Issues Related to Survey Depth 213 Susan P. Choy, MPR Associates, Inc.
Reflections on the Papers Prepared for the School and Staffing Survey Seminar Series 231 John Howard Burkett, Wilmington, NC
Contributors 1 n 241 SHOULD SASS MEASURE INSTRUCTIONAL PROCESSES AND TEACHER EFFECTIVENESS?'
Susan S. Stodolsky, University of Chicago
This paper addresses issues related to the school's educational efforts.It is in the possible inclusion of items in SASS to transactions between and among teachers, measure instructional practices and students, materials and tasks that deliberate teaching effectiveness. In order to answer efforts to educate occur. the question posed, the paper explores what purposes can be served by measuring Descriptive information about how instructional practices on a national scale. teaching and learning occur in classrooms It then examines how teaching and about what is taught provides the basis effectiveness is conceptualized. Turning for monitoring the status of instruction in a next to substantive and methodological large number of settings.Such concerns, a section describes some information can provide periodic strengths and limitations of observational assessments of stability and change in studies of classroom instruction and makes instruction, particularly as changes relate a similar assessment of survey studies. to deliberate efforts to reform or alter Some attention is then given to curricular curriculum and instruction.Similarly, if reforms and how their impact might be collected along with knowledge of assessed. The last sections of the paper particular policy initiatives, curriculum suggest specific ways in which standards, or changes in teacher instructional practices and curricular preparation or staff development, content might be measured through SASS, information describing classroom including the selection of specific school instruction can help track the impact of subjects and grade levels for attention. various policies on what transpires in classrooms.
Why Collect National Data on Descriptive information about classroom Instruction? processes also can contribute to the deliberations of teachers, teacher We begin by briefly examining some of educators, subject matter and other the main reasons to collect information educational associations, and policy about instructional practices/processes on a makers at local, state, and national levels. national scale.If we are to understand, Basic researchers also benefit from monitor, and improve our nation's schools, information about what actually goes on in accurate and timely empirical, descriptive classrooms. data about how schools work must be available. The activities that take place in Possible Limitations classrooms to engender student learning and development are the heart of any While a national picture of instruction is desirable for the reasons mentioned, it can
1 be argued that the SASS samplingstrategy linked to student dataon achievement or of few teachers in any school works other outcomes and a considerable against the utility of such information for redesign and change in samplingstrategy local (school, district) policy makers. would be required to do so.' The Many reform efforts are local and dataon question then is whether informationon instruction collected within the current instructional practices and content SASS sampling strategy might not be coverage is still useful in the absence of sensitive enough to detect local effects. data on student attainment. Weargue that These are cogent criticisms and suggest links to achievement can be direct and some alteration of the SASS sampling empirical as when teachers and their strategy to make data on instruction and students are studied, or linkscan be the effects of reform even more useful. putative and conceptual basedon known or Nevertheless, instructional data thatcan be assumed connections between practices and analyzed at national, state, and regional achievement. Existing research and theory levels seems highly useful especially since can be used to formulate the presumptive many reform programs are conducted at connections and might also inform the these levels. We will return to sampling design of empirical studies. issues in the section on recommendations. For example, studies such as Kupermintz Many argue that classroom process et al. (1995) and the IEA studies have information is most valuable when shown that content coverage is relatedto connected to student achievement and student attainment as are certain attainment. Data from NELS have been instructional processes. While our analyzed recently by Kupermintz, Ennis, knowledge is far from complete in this Hamilton, Talbert, and Snow (1995) and area, it seems safe to assume that content Lee and Smith (1995). Both research covered, particularly content that is groups found significant relationships emphasized, is more likely to be learned between certain measures of instructional than topics not taught or emphasized practices (e.g., emphasis on higher order during instruction. Thus, a description of thinking), teacher attitudes (e.g., instructional practices and content willingness to alter instructional practices coverage would allow some inferences as if students are not learning) and student to what students are likely to learn. More performance on both math knowledge in-depth and direct measurement of both (lower mental process) and math reasoning processes and student learning might be (higher mental process) items. A similar desirable, but including informationon analysis of teachers' responses to the curriculum and instruction in SASS CLAS survey by Wiley and Koon (1995) appears a useful first step. also demonstrates the potential for connections between instructional items and student attainment. How Is Teaching Effectiveness Although NCES has considered monitoring Conceptualized? student achievement in connection with The charge for this paper includesa SASS, as currently structured SASS isnot consideration of whethermeasures of
2 12 instructional practices and teaching combination of student and teacher effectiveness should be included in future behaviors in an intact lesson structure. SASS instruments.It must be noted here The model excluded behaviors that might that a broad consensus on a definition of be subject- or grade-level specific. The effective or good teaching does not exist. model did not examine student behaviors. Empirical evidence, theory and values The model adopted a transmission view of along with specified criteria for teaching.Effectiveness was effectiveness all enter into a conception of operationalized by achievement tests that effective teaching. In addition, almost exclusively contained lower-mental considerable evidence that instructional process skill items. practices need to be tailored to subject matter, developmental levels of students, Now, as evident in many standards and and other factors is now available, reform documents, a constructivist point of suggesting that effectiveness comes in a view of learning and teaching is holding number of varieties. sway. This view directs attention to students' active role in the learning Although not all teachers and policy process. Classroom arrangements such as makers endorse one view of effective group work, debate and discussion are teaching in a given time period, visions of believed integral to effective instruction. effective teaching change over time. For In addition, teaching and learning are instance, during the late 1970s and 1980s, assumed to be different from one school the process-product research program subject (or even topic) to another. No (Brophy & Good, 1986) assumed there fully general model of effective teaching were generic characteristics of good and learning is expected.Last, different teaching (i.e., they apply to all school student criteria are employed to judge subjects and grade levels considered). By effectiveness. More emphasis is placed on focusing primarily on features of teacher- reasoning, problem solving, creative centered instruction, this influential, production and long-term products. empirical research program identified a Methods of student assessment beyond number of teacher behaviors (direct standardized achievement tests are instruction model) that correlated with endorsed. student gains on standardized achievement tests in reading and math, primarily in The transmission and constructivist views elementary schools. of teaching both may have a proper place in the analysis of teaching effectiveness. Critiques of the process-product view point Flexibility in instructional strategies may out that the correlational method embodied be a hallmark of effective teaching. the existential fallacy (Stodolsky, 1988). Different instructional practices may be That is, only currently used practices could desirable depending on instructional goals enter the model of effective teaching. The and lesson formats. Instruments to assess use of data on individual teacher behaviors effective practices must contain an decontextualized instruction and made it appropriate range of items to tap lesson difficult to know how to put the structures, content, instructional strategies instructional program into operation as a and teacher and student activities. There
3
13 is a danger in only assessing popular teachers may be greater in subjects suchas visions of effective teaching whichmay mathematics, than in subjects suchas not be widely implemented or universally social studies or English (Stodolsky, appropriate at any given time. 1988). Variation is also tied to teachers' conceptions of subject matter and goals (Grossman & Stodolsky, 1994, 1995; Some Features of Classroom Shulman, 1987). Activity and Teacher Behavior: Observational Studies A variety of contextual and situational factors produce variation in teaching and Observations of instructional activity are consequently limit the stability and often thought to be the most valid method generalizability that can be expected in of data collection. While observational studies of teacher behavior, especially at studies are generally beyond the scope of the level of the individual teacher. As proposed SASS activities because theyare noted elsewhere (Stodolsky, 1990) subject very expensive, direct observation can matter, grade level, lesson type, and often be used in early stages of instrument lesson goal account for variation in development to provide relevant categories teaching behaviors and instructional and items for surveys and other arrangements. In addition, the type of measurement approaches. Observations students and track level of courses (Oakes, also have a place in validity studies and in 1985) along with district policies, type of small-scale focussed research. school, and other institutional factorsmay all influence a teacher's choice of Observational studies provide accumulated curricular content and instructional knowledge from which to formulate methods. productive questions. Useful reviews of research on curriculum and teaching are Limitations of Data from Observational provided by Shulman (1986) and Darling- Studies Hammond and Snyder (1992). Here we take a selective look at past observational Perhaps one of the most important research on teacher behavior and limitations of available observational classroom activity. studies, a feature shared with survey studies, is that the contexts studiedare Observational studies (e.g., Good & limited. The preponderance of large-scale Brophy, 1986; Good lad, 1984) have observational studies have been conducted documented a robust picture of teacher- with elementary school teachers of reading centered instruction primarily oriented and math. A few have focusedon social toward lower-level cognitive goals. studies instruction at the elementary and Recitations, variants on lecture, and high school levels (Stodolsky, 1988; seatwork are the primary instructional Newmann, 1992; Newmann & Wehlage, formats used in most classrooms. 1995). While there are a variety of small- However, systematic variation occurs scale observational studies, including when subject matter is examined. studies of classroom discourse,our Similarity of instructional practices across knowledge is not deep with respect to the
4
14 state of classroom curriculum and also important to determine the utility of instruction in fields such as science, social information obtained with other methods studies, English, foreign language or the such as teacher logs and surveys compared arts. We also have surprisingly few to observational data. observations of what actually takes place in high school classrooms. Studies of Curriculum and Another limitation of available Instructional Processes: Survey observational research is that it has Research focused primarily on teacher behaviors. A more ecological approach to classroom A number of large survey studies, often settings, such as employed by Doyle funded by NCES, NSF, and OERI, have (1983), Gump (1982), and Stodolsky provided valuable information about (1988), examines classroom activities, and curriculum and instruction in our nation's incorporates knowledge of what both classrooms. The main contributors to our students and teachers do during knowledge about curriculum and instruction, along with knowledge of instruction on a national scale are materials and tasks. However, studies of NELS:88, NAEP, Reform Up Close classroom ecology have been relatively (RUC) and SIMS. Weiss (1993) provides rare.In most observational studies, when some useful information on instructional students are observed it is to assess their practices among math and science on-task behavior or involvement. teachers. The validity of using surveys as a measurement tool in the area of Nevertheless, observations can provide curriculum and instruction has also been evidence of real instructional experiences examined (Burstein et al., 1995; Porter, unrivaled by other methods.Particularly 1995; TIMSS, 1994) and survey if one wants to understand the qualities of development is ongoing (Porter, 1995). transactions that occur in classrooms and their intellectual and social features, Useful reviews and analysis of many of observations can play a possibly unique these survey projects can be found in role.Observations, done properly, can NCES working papers (Leighton et al., reveal the connections between what is 1995), a report by Porter (1995), the work taught and how it is taught--observations of Schmidt and McKnight (1995), and can preserve classroom events as they work by Policy Studies Associates occur together. The issue for NCES or (Leighton, 1994; Leighton & Mullens, others striving for a national picture of 1994; Leighton, Turnbull, & Mullens, curriculum and instruction is under what 1994; Mullens, Weiner, Williams, & circumstances, if any, direct observation Turnbull, 1994). A catalogue of should be used as a data gathering instruments measuring the enacted approach. Clearly, observations cannot be curriculum in math and science at the done of the large number of teachers middle and high school levels is now currently surveyed by SASS. But some available (Porter & Smithson, 1995). A observational work may add to the validity list of sources for major surveys can be and richness of the SASS enterprise.It is found in the Appendix.
5
1 C The surveys distinguish between plans more accurate in reporting topic/content (intended curriculum or objectives) and coverage and emphases. actions (enacted or implemented curriculum) with the latter emphasized. The surveys are alsouneven with respect To varying degrees, thesesurveys seek to to subject matter coverage. Largely dueto measure plans (instructional goals and the efforts of NSF, majorsurvey desired outcomes), to document what is development has occurred in math and taught (content/topics and intellectual considerable attention has also been paidto processes, time allocations, emphases); science instruction. Applebee (1981, 1992) how instruction is organized (pedagogy, conducted national surveys on the teaching teacher and student activity, homework of writing and literature at the high school and tests); and resource use (e.g., level which provide modest amounts of technology, textbooks). The surveysare information about instructional practices; often described as measures of students' the Applebee work might bea starting opportunity to learn (OTL), a term point for further survey development in borrowed from the IEA studies. English along with available NELS items McDonnell (1995) provides a useful on English. According to Andy Porter, the discussion of the OTL construct. CPRE School-Based Management Survey (SBM) also contains items dealing with The uneven coverage of contexts found in instruction in language arts and social observational studies is also characteristic studies at the elementary and high school of the survey research. In an interesting levels. The items follow the four-part juxtaposition, however, most of the scheme developed by Porter and others to surveys deal with high school or eighth- assess teachers' objectives, content grade instruction while the observation covered, modes of instruction and studies are mainly at the elementary level. cognitive processes. The content items in In fact, with the exception of NAEP the CPRE surveys are rather general and fourth-grade surveys, Weiss (1994), the might provide only a starting point for Consortium on Chicago School Reform item development in English and social (1994) which borrows from NELS and studies. RUC, and the CRC (1994) survey of elementary math teachers in California, it It seems more than financial support has was difficult to locate surveys of the led to so much attention to curriculum and enacted curriculum given on a large scale instruction measures in math. at the elementary level. High school Mathematics lends itself to a systematic coverage is also somewhat uneven. NAEP analysis of its content, topics, and targets twelfth grade and therefore obtains operations because it is the best defined information primarily about advanced and probably least contentious of all school courses. There is reason to believe that subjects. Compared to other subjects, the practices used in more advanced there is considerable agreementamong courses may differ to some extent from math teachers and teacher educators about those in the earlier years of high school. best practice. Burstein et al. (1995) document that teachers of more advanced courses are
6 16 Mapping curriculum topics in other fields discussion.Studies of teachers in settings may pose a greater challenge than mapping in which reforms are under way have topics in math. Our own work (Stodolsky found a mixed picture at best. For & Grossman, 1995) on five academic example, in case studies of mathematics subjects and an analysis of English by teachers attempting to implement the Grossman (1993) and Elbow (1990), California Math Frameworks, Cohen and suggest that there is less agreement about Peterson (1990) found only modest changes content and teaching methods in subjects from conventional practice, confirming the such as English and social studies. The suggestion by Burstein et al. (1995) that difficulties confronted in developing social new practices are "layered" on to old studies and language arts curriculum ones. On the other hand, certain changes standards, confirm the lack of consensus in in math teaching such as the introduction these fields.Teachers of English and of calculators seem more widespread social studies expect considerable (Weiss, 1994). autonomy in the selection of course content, especially because they are not Research on the Coalition for Essential constrained by a perceived content Schools (Muncey & McQuillan, 1993; sequence. Science teachers also report Little, 1995) documents great variety in the freedom in choice of topics, but share a extent to which teachers adopt Coalition commitment to the scientific method. The principles. However, many teachers development of surveys with detailed report using process writing approaches topical analyses for English and social such as those advocated by the National studies presents a challenge in curricular Writing Project (Freedman, 1987; NAEP analysis and instrument development. Report Card on Writing).
In sum, significant recent efforts to To help understand the implementation of develop surveys of enacted curriculum and reforms, a first step might be to learn what instructional practices have not been teachers actually know about proposed undertaken at a level of effort similar to reforms and standards. Adequate teacher that in math and science in the fields of knowledge and understanding of reforms is English, social sciences, foreign language, far from guaranteed just because standards and other subjects including the arts. are published or new frameworks drawn Considerable new survey development, up. The CRC survey3 provides excellent particularly on curriculum topics, would examples of items used to assess teacher be required to obtain information about knowledge of curriculum frameworks.It instruction in a range of subjects and grade would also be desirable to obtain levels. knowledge of organizational support and provision of resources for reform in departments and schools.In addition, it is Curricular Reforms important to determine if teachers are asked to act simultaneously on a number Before moving on to specific suggestions of policy initiatives which may not be regarding future directions for SASS, the consistent with one another. issue of curricular reforms needs some
7 If one of the purposes in monitoring NCTM and California math frameworks instruction in the nation is to provide might carry a variety of connotations. information about the progress of Indeed, the term "reform" itself isnot curricular reforms, it must be assured that used equivalently by those reportingabout the item pool used to measure curriculum it. and instruction is adequately tailoredto the reforms advocated in each subjectmatter Last, in some cases thenew standards are studied. predicated on teacher mastery of subject matter and pedagogical content knowledge An examination of the standards for not currently widely held in the teaching curriculum in science (NRC, 1994; force. The TIMSS survey and the CRC Rutherford & Ahlgren, 1990), social survey for math teachers, include items to studies (NCSS, 1995), mathematics reveal teachers' conceptual understanding (NCTM, 1989, 1991) and of mathematical material along with English/Language arts (NCTE, 1996) pedagogy.It seems likely that items of suggests different degrees of emphasis on this type would predict student attainment, changing pedagogy and changing content. and help us document barriers to The math standards may be most explicit implementation of reforms. As such, they with respect to the vision they embody of seem important to include in any effort to pedagogy consistent with the recommended measure curricular reform. standards.4 The use of open-ended and student-generated problems and investigations which take place overa Should SASS Include Measures of number of days are examples of a Instructional Practice? constructivist pedagogy endorsed by NCTM. Specific items have been written Except for NAEP, there does not to address features of pedagogy in the appear to be any federal program in which NCTM standards; the CRC survey has instructional practices and opportunity some excellent examples. Porter (1995) to learn will be monitored in the future. reports making use of the NCTM SASS, with its large sample of teachers, standards and NSTA standards in seems an excellent vehicle for the developing opportunity to learn topic items for math and science. measurement of curriculum and instructional practices. However, the inclusion of a fairly comprehensive Specialized terminology or language set of poses items on content (e.g., as in TIMSS a possible problem in instrument or the Porter OTL four-dimension scheme) would development with items geared toward involve a lot of additional respondent time. reforms. Burstein et al. (1995) in their Further, to adequately monitor pedagogy validity study of math instruction items and track reforms, additional items would found that teachers did not always interpret be needed. terms in the same manner (e.g., "math modeling" had a number of different Since NAEP is an ongoing meanings to the teachers they studied). A program that taps into curriculum and instruction in term like "investigations" used in the a number of school subjects (although maybe
8
1 8 not very deeply), an optimal plan for topical analysis is also available in U.S. SASS would complement and supplement and world history, although not the efforts planned under NAEP. Some broader social studies. High school English school subjects and grade levels not is not mapped in much detail nor is the regularly covered by NAEP should be elementary school curriculum. (Exceptions included in SASS. At the same time, are the three-dimensional content structure more targeted efforts to link with NAEP developed by Freeman, Porter and others and/or assist NAEP to enhance its for fourth-grade mathematics and some curriculum and instruction measures would items from NAEP dealing with reading be highly desirable. Some links to NAEP and writing instruction). The four- would also provide tie-ins to student dimension topic items such as developed survey responses about their instructional by Porter (1995) for his recent OTL study experiences. for math and science, seem a suitable model for item sets to be used in SASS. SASS seems ideally suited to monitor the The four dimensions include two classroom consequences of reforms such as dimensions of topics and the degree of curriculum standards.(It may be asking emphasis each receives, cognitive activities too much to monitor the myriad of other (with time distribution) and the medium reforms under way.) A selection of (mode) of instruction (with time specific school subjects and grade levels distribution). For school subjects other seems the best strategy here. However,in than math and science, item development order to maximize insight into how analogous to the Porter model would be reforms work, it would be desirable to needed. have more teacher respondents from a given school than has been the case in Pedagogy previous SASS sampling, so that information about the presence of There are quite a few items and item types particular reform efforts in the schools dealing with pedagogy or teaching methods could be obtained. Linking with NAEP that seem applicable to most subjects and under selected circumstances would also grade levels, although a careful analysis benefit from more clustering of teachers in would be required to assure that practices schools. found in elementary school classrooms were adequately sampled. The language in which methods are described might also Item Selection require modification and field testing when applied in contexts other than those Curriculum Content previously surveyed. In addition, specialized language from reform Let us begin by examining measures of documents should be used with caution and content taught. As discussed earlier, there fully pilot tested to assure common is a substantial pool of items to use in understandings. measuring the content taught in science and mathematics, especially at the middle As Leighton, Turnbull, and Mullens (1994) school and high school levels.Limited note, subject-specific questionnaires have
9
19 been the rule recently. Manycommon Items to Track Curricular Reforms instructional items reappear insurveys for teachers of different subjects in addition to In addition to content/topic items and specific items for each subject. The 1994- pedagogy items, new items should be 95 SASS Follow-up Teacher Questionnaire developed that assess teachers' specific has a number of sections dealing with knowledge of reforms. The CRCsurvey teaching methods that are promising and provides some good examples of such which build on development work from items for the California Math Frameworks. other surveys we have discussed. Whilea Teacher's subject matter and pedagogical good starting point, a careful review content knowledge required for should be made for appropriateness to implementing reform should also be grade levels and school subjects selected measured.' A particularly promisingitem for study. Also, there may besome format has been used in TIMSS and the overlap in constructs if four-dimension CRC survey, among others. The items content items such as those in Porter are ask teachers to envision an instructional also in the survey. sequence of lesson parts used to teach a specified topic. For example, the CRC Goals survey asked questions about instruction dealing with fractions in anopen-response Burstein et al. (1995) recommend against format. The TIMMS items aremore the inclusion of items measuring goals,as structured. These items tap lesson they did not find a good match with organization, content emphasis, responses and other data sources such as pedagogical content knowledge and subject the goals inferred in tests or teacher matter knowledge and may be an effective assignments. On the other hand, they did way to tie together features of instructional find meaningful relationships between processes and content in a manner that endorsement of reform goals and reform approximates what actually happens in practices, but not between endorsement of classrooms. traditional goals and traditional practices. This issue would seem to require further Teacher Attitudes, Professional Activities, study before eliminating goal items from and School Culture national surveys. The data pattern suggests in part that most teachers believe In creating item sets for a survey, it would traditional goals are worthwhile,even be desirable to include measures of teacher those who are moving their practice in the efficacy and willingness to adapt direction of reform. This findingseems instruction as these scales have important another example of the tendency of predictive power in connection with other teachers to add on to their practice without instructional items. Professional giving up old patterns. Thus,some development activities and participation in tensions inherent in changemay be subject area and other networks should revealed effectively through analysis of also be assessed. Items thatassess the goal items. Although not the highest extent to which the school culture and priority, if respondent time allows, goal organization support reformare also items should be retained. useful. Basic information suchas whether
10
20 a school or department has officially While targeting teachers of certain subjects adopted a particular reform should be and grade levels to answer survey items on collected. Taken together, these items curriculum and instruction is the would reflect teachers' opportunities to recommended approach, the decision learn about or deepen understanding of regarding what school subjects and grades new approaches and to gain support in to select is not an easy one. We thus trying to implement reforms. These scales recommend a mixture of large and small could be part of the teacher background studies on instruction and teaching section of the survey. effectiveness within the SASS program. When the state of the art is adequate in To create respondent time for the terms of prior instrument development, suggested content/pedagogy and teacher larger numbers of teachers should be knowledge items, we suggest two studied. When the state of the art is less strategies. One is to eliminate certain adequate, small numbers of teachers parts of the current SASS survey since should be studied in the service of items have been administered over many instrument development and validation. years and may be given to a subsample or less frequently.In particular, items Subject Areas and Grade Levels dealing with teacher control over policies such as discipline, hiring of new teachers Because so much investment has been could be omitted. The list of perceived made in instrument development in math, problems (poverty, tardiness, etc.) might and because the NCTM standards were in also be eliminated or given to a subsample the vanguard, it seems appropriate to use of teachers. The second strategy takes us math as one of the target subjects. The into the realm of sampling to which we scope of surveys about math should be now turn. expanded to include math in the upper elementary grades (4-6) along with middle school and high school.If costs permit, Who Should Be Surveyed? middle and high school science is another area in which some useful instruments are We believe that all SASS respondents need available. not answer all survey items. We recommend the use of item/person In addition to math, Porter (1991) suggests sampling in the administration of SASS. English as an important understudied area. We recommend selecting teachers of Leighton, Turnbull, and Mullens (1994) certain grades and school subjects to suggest history as another possibility. At respond to the curriculum and instruction the middle and high school levels, both are survey. Other respondents could be used plausible options, with history having to answer more general questions from somewhat of an edge in terms of existing SASS. In addition, even teachers within instrumentation.If history was selected as the recommended grades and subjects a focal subject, the elementary grades could be directed (say, by use of their should again be included.Fifth grade is birthday as a sorting mechanism) to typically the year U.S. history appears in answer only certain parts of the survey. the elementary curriculum. So surveys in
11
9 f1/4:0 J3_ grades 4-6 would make sense. However, interdisciplinary teaching.It seems the elementary social studies curriculum is important to bear this in mind in reviewing quite diverse and content items should items for inclusion in SASS and in range well beyond history to articulate thinking about how to select teachers. with actual practice. One hopes that there are teachers whose instructional programs are strongly Further, within social studies, there is integrated for whom answeringa more contention about the direction the subject conventional survey could be problematic. should take. Both history and social studies Such teachers may be more often found in standards (NCHS, 1994; NCSS, 1995) elementary schools, but high school have been formulated and there is programs emphasizing subject integration considerable tension among adherents to are also being implemented. each set of standards. A SASS survey aimed at charting reform in this area, Instrument Development, Pilot Studies, would be challenged to accommodate Validity Studies differing points of view. Support for survey instrument development English/Language Arts standards have seems in order as an important step to been released recently (NCTE, 1996) but prepare for the next SASS cycle. These offer little guidance with respect to content efforts should be directed at enhancingour coverage as they emphasize pedagogy. In capability to measure curriculum and some cases, English or Language Arts instruction in subjects hitherto consists of instruction primarily geared to understudied--especially elementary math, developing skills in reading and/or writing; and history/social studies at the in other instances the instructional program elementary, middle and high school level. isdirected more toward literature. Based Work might also begin on mapping the on all these factors, the choice of English/language arts curriculum for future history/social studies for inclusion in SASS inclusion in SASS. might have a slight edge. In any case,we envision smaller scale studies in subjects We have already discussed the types of beyond math and science oriented items needed to assess the implementation primarily toward instrument development of curricular reform.(See section on and validation. Curricular Reforms.) Item developmentor modification of existing items should also A Caution go forward in preparation for the next cycle of SASS. A cautionary note should be sounded with regard to the subject-specific focus of this The instrument development projects discussion. Current instruments andour would involve multi-method investigations discussion have assumed that instruction is that could determine the validity of pilot compartmentalized by subject.Empirical items and other methods. Effectiveuse of evidence suggests this is still largely true, teacher logs, collection of teacher but a number of curricular reforms call for assignments, exams and other materials; more subject matter integration and textbook analyses, and classroom
12 22 observations might be incorporated in the Brophy, J., & Good, T. L.(1986). instrument development process.If new Teacher behavior and student achievement. item sets are ready for administration in In M. C. Wittrock (Ed.), Handbook of SASS, we recommend smaller sample research on teaching, 3rd edition. New studies in the first round so that their York: MacMillan. validity can be established. We agree with Burstein et al. (1995) that validation studies Burstein, L., McDonnell, L., Van Winkle, should regularly accompany the J., Ormseth, T., Mirocha, J., & Guiton, introduction of new surveys. Thus G. (1995). Validating national curriculum continuing validity studies should be indicators.Santa Monica, CA: RAND. supported during SASS administration. Choy, S. P., Henke, R. R., Alt, M. N., Additional small studies conducted through Medrich, E. A., & Bobbitt, S. A. (1993). SASS (perhaps in Follow-Up surveys) Appendix C: Technical notes.Schools might delve into topics of interest to the and Staffing in the United States: A nation from time to time.Illustrative is statistical profile, 1990-91. Washington, the section of the SASS 1994-95 Teacher DC: U.S. Department of Education, Followup Questionnaire which inquires National Center for Education Statistics. about portfolio assessments. Inquiry into special topics such as this could be a Cohen, D. K., & Peterson, P. L.(1990). regular part of SASS, with only a fraction Special issue of Educational Evaluation of teacher respondents being asked to and Policy Analysis, 12(3), 233-353. provide information. In this manner, not all teachers would take exactly the same Darling-Hammond, L., & Snyder, J. set of items, but reliable information could (1992). Curriculum studies and the still be obtained on a number of interesting traditions of inquiry: The scientific issues. tradition.In P. W. Jackson (Ed.), Handbook of research on curriculum. New York: MacMillan. References Davis, C., & Sonnenberg, B. (Eds.). Applebee, A. N. (1981). Writing in the (1995). Programs and plans of the secondary school: Current practice in National Center for Education Statistics English and the content areas.(Research (NCES 95-133). Washington, DC: U.S. Report No. 21). Urbana, IL: National Department of Education, National Center Council of Teachers of English. for Education Statistics.
Applebee, A. N.(1992).Literature in Doyle, W. (1983). Academic work. American high schools. Albany, NY: Review of Educational Research, 53, 159- Center for the learning and teaching of 199. literature. Elbow, P. (1990). What is English? Urbana, IL: National Council of Teachers of English.
13 23 Freedman, S. W. (1987). Response to Kupermintz, H., Ennis, M. E., Hamilton, student writing. Urbana, IL: National L. S., Talbert, J. E., & Snow, R. E. Council of Teachers of English. (1995). Enhancing the validity and usefulness of large-scale educational Good lad, J. I.(1984). A place called assessments: I. NELS:88 mathematics school. New York: McGraw-Hill. achievement. American Educational Research Journal, 32(3), 525-554. Grossman, P. L.(1993). English as context: English in context. Working Lee, V. E., & Smith, J. B.(1995, Paper Series, Center for Research on the Summer). Effects of high school Context of Secondary School Teaching restructuring and size on gains in (CRC), Stanford University. achievement and engagement for early secondary school students. Sociology of Grossman, P.L., & Stodolsky, S. S. Education. (1994). Considerations of content and the circumstances of secondary school Leighton, M. S.(1994). Measuring teaching.In L. Darling-Hammond (Ed.), instruction: The status of recent work. Review of research in education: Vol. 20 Washington, DC: Policy Studies (pp. 179-221). Washington, DC: Associates. American Educational Research Association. Leighton, M. S., & Mullens, J. E. (1994). Measuring curriculum content: Grossman, P. L., & Stodolsky, S. S. The status of recent work. Washington, (1995). Content as context: The role of DC: Policy Studies Associates. school subjects in secondary school teaching. Educational Researcher, 24(8), Leighton, M. S., Turnbull, B. J., & 5-11. Mullens, J. E. (1994). Measuring opportunity to learn: Advancing the state Guiton, G., & Oakes, J.(1995). of the art.Washington, DC: Policy Opportunity to learn and conceptions of Studies Associates. educational equality.Educational Evaluation and Policy Analysis, 17(3), Little, J. W. (1995).Subject affiliation in 323-336. high schools that restructure. In L. S. Siskin & J. W. Little (Eds.), The subject Gump, P. V.(1982).School settings and in question: Departmental organization their keeping.In D. L. Duke, (Ed.), and the high school. New York: Helping teachers manage classrooms. Teachers College Press. Alexandria, VA: Association for Supervision and Curriculum Development. McDonnell, L. M. (1995). Opportunity to learn as a research concept and a policy instrument. Educational Evaluation and Policy Analysis, 17(3), 305-322.
14 24 Mullens, J. E.(1995). Classroom National Council of Teachers of English instructional practices: A review of (NCTE). (1996).Standards for the existing measurement approaches and their English Language Arts. Urbana, IL: applicability for the Teacher Followup National Council of Teachers of English. Survey (NCES 95-15). Washington, DC: U.S. Department of Education, National National Council of Teachers of Center for Education Statistics. Mathematics (NCTM). (1989). Curriculum and evaluation standards for Mullens, J. E., Leighton, M. S., Turnbull, school mathematics. Reston, VA: B. J., Weiner, L. K., & Williams, A. S. National Council of Teachers of (1995). Measuring instruction, curriculum Mathematics. content and instructional resources: The status of recent work (NCES 95-11). National Council of Teachers of Washington, DC: U.S. Department of Mathematics (NCTM). (1991). Education, National Center for Education Professional standards for teaching Statistics. mathematics. Reston, VA: National Council of Teachers of Mathematics. Mullens, J. E., Weiner, L. K., Williams, A. S., & Leighton, M. S.(1994). National Research Council.(1994).Draft Measuring instructional resources: The national science education standards status of recent work. Washington, DC: (summary). Arlington, VA: National Policy Studies Associates. Science Teachers Association.
Muncey, D. E., & McQuillan, P. J. Newmann, F. M. (1992).Higher-order (1993).Preliminary findings from a five- thinking and prospects for classroom year study of the Coalition for Essential thoughtfulness. In F. M. Newmann (Ed.), Schools. Phi Delta Kappan, 74(6), 486- Student engagement and achievement in 489. American secondary schools (pp. 62-91). New York: Teachers College Press. National Center for History in the Schools (NCHS). (1994).National standards for Newmann, F. M., & Wehlage, G. G. United States History. Los Angeles, CA: (1995).Successful school restructuring: A Author. report to the public and educators by the Center on Organization and Restructuring National Council for Social Studies of Schools. Madison, WI: University of (NCSS) Task Force on Standards for Wisconsin. Teaching and Learning in the Social Studies.(1995). A vision of powerful Oakes, J.(1985). Keeping track: How teaching and learning in the social studies: schools structure inequality. New Haven: Building social understanding and civic Yale University Press. efficacy. Washington, DC: National Council for Social Studies.
15 Porter, A. C. (1991). Creating a system Shulman, L.(1986). Paradigms and of school process indicators.Educational research programs in the study of Evaluation and Policy Analysis, 13(1), 13- teaching: A contemporary perspective.In 29. M.C. Wittrock (Ed.), Handbook of research on teaching, third edition. New Porter, A. C. (1995). Developing York: MacMillan. opportunity-to-learn indicators of the content of instruction (Progress Report). Shulman, L.(1987). Knowledge and Madison, WI: University of Wisconsin- teaching: Foundations of the new reform. Madison, Wisconsin Center for Education Harvard Educational Review, 57, 1-22. Research. Stodolsky, S. S.(1988). The subject Porter, A. C., Kirst, M. W., Osthoff, E. matters: Classroom activity in math and J., Smithson, J. L., & Schneider, S. A. social studies.Chicago: University of (1993). Reform up close: A classroom Chicago Press. analysis (Draft Final Report to the National Science Foundation on Grant No. Stodolsky, S. S.(1990). Classroom SPA-8953446 to the Consortium for Policy observation.In J. Millman & L. Darling- Research in Education). Madison, WI: Hammond (Eds.), The new handbook of University of Wisconsin-Madison, teacher evaluation: Assessing elementary Wisconsin Center for Education Research. and secondary school teachers. Newbury Park, CA: Sage. Porter, A. C., & Smithson, J. L. (1995). Enacted curriculum survey items Stodolsky, S. S., & Grossman, P. L. catalogue: Middle school and high school (1995). The impact of subject matter on mathematics and science (Catalog for the curricular activity: An analysis of five U.S. Department of Education, Office of academic subjects. American Educational Educational Research and Improvement, Research Journal, 32(2), 227-249. National Center for Education Statistics). Madison, WI: University of Wisconsin- Weiss, I. R.(1994). A profile of science Madison, Wisconsin Center for Education and mathematics education in the United Research. States:1993. Chapel Hill, NC: Horizon Research, Inc. Rutherford, F. J., & Ahlgren, A. (1990). Science for all Americans. New York: Wiley, D. E., & Yoon, B.(1995). Oxford University Press. Teacher reports of opportunity to learn: Analyses of the 1993 California Learning Schmidt, W. H., & McKnight, C. C. Assessment System (CLAS). Educational (1995). Surveying educational opportunity Evaluation and Policy Analysis, 17(3), in mathematics and science: An 355-370. international perspective. Educational Evaluation and Policy Analysis, 17(3), 337-354.
16
26 Sources of Surveys
Teacher and Student Questionnaires, Schools and Staffing Survey:1993-94. Spring.1994. Madison, WI: Center on 1993. Washington, DC: National Center Organizing and Restructuring of Schools. for Education Statistics.
Charting Reform: The Teachers' Turn, Schools and Staffing Survey Teacher 1994. Chicago, IL: Consortium on Followup Survey Questionnaire for Chicago School Research. Current Teachers: 1994-95.1994. Washington, DC: National Center for CPRE Reform-up-Close Study.1989. Education Statistics. Madison, WI: Consortium for Policy Research in Education. Teaching and Learning Conditions of the School and Classroom.1995. Policy CPRE Upgrading Mathematics Study. Studies Associates. Washington, DC: 1992. Madison, WI: Consortium for National Center for Education Statistics. Policy Research in Education. Third International Mathematics and CPRE School-Based-Management Study. Science Study.1994. Chestnut Hill, MA: 1993. Madison, WI: Consortium for TIMSS Study Center. Policy Research in Education. Validating National Curriculum Indicators. CRC (Center for Research on the Context 1993. Santa Monica, CA: RAND. of Teaching).1994. Survey of elementary mathematics education in California. Teacher Questionnaire.
NAEP The Nation's Report Card.1993. Washington, DC: Office of Educational Research and Improvement.
National Education Longitudinal Study of 1988 (NELS).1988. Washington, DC: National Center for Education Statistics.
17 27 I. This paper was written at the request of Policy Study Associates foruse by NCES. The author is solely responsible for the ideas presented.
I would like to acknowledge the generous assistance of John Mullensat PSA in providing numerous documents and surveys. Andrew Porter and Fred Newmann, bothat the University of Wisconsin, Madison, Leland Cogan of the TIMSS at Michigan State University, Joan Talbertat CRC at Stanford University, and Valerie Lee at University of Michigan, also madesurveys or papers available.
Revision of the paper benefitted from the comments of Dominic Brewer and Cathleen Staszof the Rand Corporation and Pamela Grossman of the University of Washington.
I have tried to cite sources as appropriate throughout the paper. However, thesurveys and papers that form the core used in preparing this paper are much like an extended family. Surveyshave uncharted historical connections to one another, with items borrowed and adapted freely. Manycommonalities in thinking appear in papers on the topic.I apologize in advance for any omissions in citations or for mistakenly citing a source that is not the definitive one.
2. Phillip Kaufman in this series suggests linking SASS to a new NELS.
3. The CRC survey of California elementary math teachers uses some items from the National Centerfor Research in Teacher Education at Michigan State University.
4. Of course the NCTM standards also recommend change in what is taught in math classes.
5. Resource use, especially what textbooks and other materials are used, is beyond thescope of this paper. The omission does not reflect a lack of importance.
18 28 TOWARD AN ORGANIZATIONAL DATABASE ON AMERICA'S SCHOOLS: A PROPOSAL FOR THE FUTURE OF SASS, WITH COMMENTS ON SCHOOL REFORM, GOVERNANCE, AND FINANCE'
David P. Baker, The Catholic University of America and American Institutes for Research
The question before us is what could, and schools.It has, in many ways, already should, NCES's Schools and Staffing become this, but this goal should be made Survey (SASS) become in the future? In more explicit as the survey's main other words, what role should subsequent objective. The focus of future SASS's collections of SASS play in NCES's should be broader than in the past by statistical program? It is recommended including a wider scope of information here that, while some continuity between about how American schools are past and future surveys is important, SASS organized.It should become the baseline, should not just repeat what it has done in fundamental survey for a host of NCES's the last three surveys. efforts in describing elementary and secondary schooling in the country. SASS After three data collections and almost of has moved in this direction and it should decade of service to NCES, SASS is at a move further still beyond a more narrow cross-roads in its development. One road survey on teacher supply and demand to an leads on in the same direction set by the omnibus survey of school organization. three completed surveys, with perhaps only some minor additions and slight I recommend a renewed SASS for four modifications to the items. Although I see reasons that are explored in some detail nothing wrong with doing reports similar below. The reasons involve: (1) what to past ones with fresh data, it is SASS has become over the past decade; nevertheless a limited strategy that will not (2) what I perceive as some critical weak maximize the use of future SASS's. To points in NCES's statistical program on K- essentially repeat what has already been 12 schooling that need addressing; (3) done retards emerging developments SASS what knowledge and technical capabilities has made over the past three surveys. The about surveying schools have been accrued other road follows these developments over the last three SASS's; and (4) what I towards a new SASS that presents NCES think NCES should collect about how with an opportunity to provide more schools approach reform, their governance complex and broader information on structures, and finances in the American schools as educational organizations.I, system of formal education. therefore, recommend taking this other road, one that leads in a new direction. Before discussing the reasons for a new SASS, I will first describe what a broader The new direction is to make SASS organizational survey might look like. NCES's main vehicle for organizational This description will move through the information about the nation's K-12 general to the more specific. Then I
19 n outline one example of how this might be time and so forth. The focus on each of done within the current length and scale of the four components should be on how SASS. This is followed by discussions of decisions are made, what decisions have a new SASS in comparison to the current recently been made and how they are one; SASS and organizational theory; and reviewed and changed. The key the reasons why a new SASS is useful at information to capture is organizational this point in NCES's history. process not just organizational structure. During the planning forum some suggested that this kind of information is too The New SASS as an complex to capture in a survey format.It Organizational Database on is true that this is not simple information, Schools but neither are the types of information that SASS currently measures. There is a What would an organizational database host of detailed research on these aspects consist of and how different would this of school organization that a survey could new SASS be from the last three? What make use of to develop short, but accurate, basic organizational components should be indicators of these components. Examples added to the existing organizational of information needed about each information collected in the 93-94 SASS? component are as follows. What essential information about a school should SASS collect for NCES? To Teachers. SASS has certainly been a answer these questions, I see at least four major source of information on K-12 main perspectives that an organizational teachers, and a lot of this should be kept survey of schools must take to maximize in a new SASS. But, as I illustrate below, its benefit for NCES. some of the teacher information will need to be scaled down to make more room for 1. One Main Focus of the Survey Must Be information on the other three central on How a School Organizes Its Main components of school organization. But Dynamic Components still we need to know more about how teachers and their activities are controlled Information should be collected on how a (or not) in schools, who does the school organizes its four main components controlling, and over what issues. We of: (1) faculty; (2) student body; (3) also need to know more about what things curriculum/instruction; and (4) immediate teachers have real decision making power outside environment such as parent and over and whether or not it is used. Some community groups interested in education. of this kind of information was collected in By "dynamic" I mean more emphasis on all three SASS's, but more direct items how decisions and control move through need to be developed than the questions the organization and less emphasis on about general influence teachers (and static titles and fixed structures.It is others) might have on certain issues. The better to spend limited questionnaire space key here is to capture a picture of how on, for example, how a departmentalized much administrative control there is over system influences control over teachers teaching and the work conditions of than on information about the status of teachers and how much decision-making department heads such as part-time, full- power teachers have within the school.
20
30 For example, are teachers and their There are many such issues to choose from teaching observed, inspected, and and not all can be included, but again the assessed? By whom and with what key here is to gain a picture of how a purpose? What kinds of collegial control school makes decisions about students and is there in the school versus more programs for students. Each prior SASS hierarchical control from non-teachers? has increasingly done some of this, but Measures of teacher autonomy and school- more explicit information on these kinds of based management are crucial to collect. things should become part of the new SASS. Students. As the recent NCES Fast Response Statistical Survey (FRSS) on Curriculum and instruction. A school's curricular tracking and student assignment curriculum is a central and complex to courses shows, NCES's student-level component of its educational operation. data sets such as HS&B and NELS do not Yet, one of the weakest points within tell us many key things about how schools NCES's overall statistical program is organize student (NCES, 1994; Baker, information on what gets taught and how it Ralph & Manlove, forthcoming). SASS gets taught. Most of the information on could play a very important role in what is taught in schools is inferred from collecting information about how schools individual student course taking records or manage their students. A short list of from teacher teaching loads; little is examples include information on: known about national estimates of school organization of the curriculum through Policies about student choice of secondary school courses or coverage time courses at the middle and secondary in the elementary school. In the past, school level NCES has collected relatively little information on curriculum, however there Student discipline policies have been improvements. The attention that the term "Opportunity to Learn" has Ways in which the school organizes brought to curricular aspects of schooling parental involvement in schooling will probably increase the amount of information collected on curriculum and Decisions about programs for instruction in the future (Brewer & Stasz, students and policies about access 1995). A new organizational SASS would to these programs be a perfect vehicle from which to gather basic OTL information on curriculum and School policies and management instruction at the school level. For philosophies towards student example, from just one item on the fore abilities mentioned FRSS on curriculum tracking, NCES received for the first time national Pedagogical approaches to variation estimates of how public schools organize among students in motivation, their course structure, requirements and intelligence and educational and enrollments in tenth grade mathematics occupational goals (i.e., tracking and English. This kind of information and ability grouping) should be collected for other grades. Besides course structure, information
21
'A 1 should be collected about curriculum Other components.Certainly there are content, such as what is meant by an other parts to a school that makeup its "algebra course" in the eighth grade for total organizational picture. The most example. Some parallel information on obvious one without its own section above how the curriculum is implemented by is the school's administration. In part this teachers would be equally useful. Also is because the administration playsa role information on how local, state or national in organizing and managing each of the standards are translated into actual four main components of the school,so it curriculum is important to collect, as well is included in this way. Additionally, as information about grading policies and other information about the structure and how decisions in these areas are made at function of the administration would be the school level. Some information of this essential to complete the organizational type exists in SASS, but it is oriented description of a school. Theremay be towards representing a teacher's teaching other organizational components thata new load, not the school's organization of its SASS could include, but I have tried to full curriculum. NCES should know the provide the essential minimum. basic dimensions of the K-12 grade curriculum in the nation's schools and a 2. A Second Main Focus of the Survey new SASS is a good way to do this. Should Be on How Much of the School's Because of the complexity of curriculum, a Organization is Influenced by District, State new SASS should not attempt to do an in- and Federal Educational Agencies depth collection on this area of schooling, but some basic information would go a One of the unique features of the long way toward improving NCES's American educational system is its local reporting on curricular and instructional administration of schools. From the information. beginning of American formal education, communities organized and controlled their Immediate outside environment. Schools are own schools. This arrangement carried linked to the community they serve over into the period during which an through a series of semi-formal extensive public school system was organizations such as parent organizations, constructed by local governments. The business councils and other community American public school then is a local groups. Schools interact with these and entity. However, over the past century have varying roles in their creation and different levels of government have takena influence on education in the local larger role in the regulation and control of community. Although these kinds of schooling. Local education agencies organizations may not be as fundamental (LEAs) are influenced by other to the day-to-day operation of a school as government agencies at the state and teachers, students and curriculum, they do federal levels. Added to this is the play a significant role in school variation across the country as to which organization. A new organizational SASS different levels of government have control should learn more about how schools over which specific areas of schooling. connect with these kinds of organizations. There is also a sizable private school sector at both the elementary and secondary levels which, compared to private schooling in many other developed
22 nations, is relatively un-regulated by It is important, then, for a survey of government. school organization to be able to gain some information on the influence that a Taken together these features make the multi-governance structure has on schools American school system local, varied and what they do on day-to-day basis. across place, and multi-governed. And Currently SASS includes some questions this has direct and important consequences. about influence from the district in some For example, it may be far more difficult broad areas, but as stated above, these are to standardize education within the U.S. not very specific and do not help to really than elsewhere. This feature is often put distinguish between different kinds of forth as one reason why the width of the governance environments found around the distribution of academic performance country and across public and private among American students is large sectors. The new SASS should include compared to that found among students in information about how much influence other countries. Also it is not necessarily various levels of educational governance easy to determine what is occurring within have on the way a school organizes the schools in this kind of a system. Indeed four main components described above. one of the reasons for why NCES gathers most of its information through surveys This is particularly true of the district for instead of central administrative records, public schools and other similar supra- as is commonly done in more centralized administration of some private schools education systems of other countries, is such as the diocese for the Catholic because the unique governance structure of schools. M. Ross' paper in this collection American schooling. focuses on school districts. Many of the managerial issues that I outline above have The suspicion, at least, is that there could their origins in district offices even though be large variation in schooling across the implementation and control are at the nation. Some assume this and hold it as a building level. Some information on positive in the form of high potential for policy setting and school control by the innovation in this kind of a system; hence, district is essential to complete the picture for example, the reform notion of further of the organizational nature of a school. decentralization of an already relatively But even having said this, I still decentralized system. Others present this recommend that the primary unit of survey more negatively and suggest that focus be the school for the reasons that I parochialism and incompetence in school list below. administration can often go unnoticed and unchanged in this kind of a system. Although this is a complicated area, NCES Regardless of which view is more should at least know some of the basic accurate, this unique governance structure levels of influence that district, state, and is a fundamental feature of American federal agencies have on schools. And schooling and NCES needs to provide as this needs to be more specific than the much information on this feature's impact general impression of the principal in on schooling nation-wide as is possible. terms of global influence, as is now asked in SASS. For example, which kinds of schools are bound by district rules and
23 a procedures for the hiring of teachers, array of other instruction personnel, such assessments of instruction, creation of as curriculum specialist, resource teachers student programs, curricular issues andso and other classroom level specialists, forth? How much state control and which adds to the total instructional regulation reaches the school in these resources reaching the students.Optimally areas? How much federal regulation NCES should know what portion of reaches the school and what form does it students and teachers have access to these take? resources and how many of these actually make use of them.Additionally it would 3. SASS Must Gather More Informationon be useful to know the degree to which Financial Resources and Their Flows to the resources are devoted to developing Point of Instruction curriculum and instruction in the school and to parent and community organizations Resources available to schools, broadly connected to the school. Even information defined, will increasingly be a topic of on very basic instructional resources, such policy debate as funds for education meet as supplies and simple technologies, could with stiffer competition for other activities. be useful information about overall Although a new SASS can not and should resources and school organization. not replicate the detailed school finance work already being done by NCES, it 4. A New SASS Must Include Measures of should include enough information on School-level Educational Outcomes costs and resources to make some estimates of how schools manage The utility of what is proposed here for a resources. The current information on new organizational SASS would be greatly teacher salary schedules is important to compromised if it did not somehow retain since it helps to generate cost include information on educational estimates.' outcomes. Even though the details of this are explored elsewhere in this planning Since for the public sector at least, most process, I wanted to make my separate finance and its controls are set beyond the recommendation known. For numerous school, both NCES's F33 at the local reasons, NCES needs to incorporate into district and LEA level and the NPEF SASS school-level information on student survey at the state level capture basic achievement, promotion, dropout, national financial information. What is disciplinary actions, college application missing is how much financial resources and so forth.Currently SASS does a are transmitted into school level resources small amount of this in characterizing a and how this is done. How much of the school's student body, but more should be overall resources flow to the point of done in the next SASS. The central issue instruction? Besides teacher salaries and that reoccurs within the American quality, one additional way SASS could educational policy arena is what outcomes add organizational information on this is to are associated with what organizational examine what instructional resources features. From macro "school effects" teachers have access to within sampled research to work on the micro schools. For example, in more affluent improvement of instruction, the districts, teachers can be assisted by an assumption is made that schools and
24
34 outcomes of students are linked. Some an omnibus, organizational SASS that take a more explicit organizational view of included educational outcomes, NCES this assumption, others are more skeptical, could make a major contribution to the but nevertheless this idea is the backbone debate about what kinds of schools of most education policy. So for NCES to produce what kinds of outcomes and maximize its impact on the public perhaps add insight on how this comes discourse on education, it should maximize about. its ability to provide information related to this most basic of policy assumptions. I am not necessarily arguing for a full Certainly both the NAEP and assessment of students in each surveyed HS&B/NELS surveys yield important school in SASS. Perhaps some sub- outcome information for NCES, but sampling could be done; or some form of neither has the perspective that a new linking with NAEP; or even some use of SASS could provide. NAEP was not extant student assessments within schools originally designed to examine any inputs could be made. But the point is that for to achievement beyond technical controls SASS to become a central vehicle for for assessment estimates. And the NCES, along with NAEP and NELS, it HS&B/NELS surveys contain some school must include information on school organizational data and student outcomes, outcomes. neither is as extensive as what is proposed here.Also, the main focus of these longitudinal surveys on the student's How Could a New Organizational progression through school and into the SASS Be Implemented? labor market shapes the survey to answering more questions about what What is proposed below is just one influences individual achievement and example, a sketch really, of what might be attainment than what types of school done to incorporate the ideas above to organization yield what types of outcomes make SASS an omnibus, organizational for their students. survey of K-12 schools.In doing this I make several assumptions. This is perhaps a fine distinction, but one that I think is worth considering. We First, I assume that the next round of actually do not know much about how SASS will have to be designed within various differences in school organization roughly the same length and response influence student outcomes in the burden parameters as was in the 1993-94 aggregate. This was, for example, the survey.This makes planning additions original intent of the first Coleman report and changes more or less a zero sum (1966), but the study used what would operation. For something to be added, now be considered a very simplistic notion something of equal size needs to be of school resources and almost completely deleted.It also makes the job of designing ignored the organizational dimension of a new approach of SASS difficult.This is schools. Thus when family and student particularly so for me since I find many of background effects were found to "swamp" the current SASS items useful and I do not most school level effects, this line of easily part with them. research went in another direction. With
25 35 Second, I assume that the school will be Finally, for this exercise, Iassume that the the main unit of focus of a new libraries component stays asa separate organizational SASS. Information from component and the Teacher Listing Form supra-units such as districts and other stays as a way to generate any teacher LEAs (private or public) would be mostly sample. The question then is what might focused on the sampled school. The same be done with the Teacher Demand and would be true for teachers in that the focus Shortage/District, Principal, School, and would be on their experience with this School Teacher questionnaires to make school, as is currently the case with some room from items that capture the ideas parts of the teacher questionnaire. The above? logic here is the same as in HS&B or NELS with one primary unit of analysis Teacher Demand and Shortage/District with supporting information from others, Questionnaire but unlike these student level surveys, the focus of a new SASS is explicitly on the The Teacher Demand/District school.3 questionnaire should be less oriented towards general district programs and Third, I assume that most of the basic more towards whether or not the district questions about schools such as sets policy and procedures on issues that enrollments, location, school types and so correspond to the management of teachers, forth (i.e., many of things collected in students, curriculum, and other parts of section A of the School Questionnaire) the school. These items should be geared would remain. Although the SASS staff toward the district's school in the SASS might want to consider more efficient and sample, not just in general terms about all less burdensome ways to generate this schools in the district. Some new items from principals. For example, would it be will need to be developed. For new items more efficient if principals were given a and for many items currently on this listing of their school's basic information questionnaire what needs to be added is from CCD that were used to create the some clear indication of the relative sample and be asked to verify the accuracy control the district (or appropriate LEA) of these? has over the target school vis-a-vis selected areas of school operations. The Fourth, let us assume that approximately items should indicate where the four to five items per revised questionnaire relationship between the district and a about each of the four components school is on a range of control from one of described above would provide enough "district set enforced policy" to "policyas information to draw an effective picture of a guideline with considerable school-level the organization of the school. discretion." Also, this should be the place where some information is gathered about Fifth, I assume that what is proposed here the influence of state policies and federal can be adapted for schools in the private policy impact. sector and Indian schools.Also, there will need to be some specific adaptations for Specifically, I would recommend that elementary and secondary schools. section A, B, and C of this questionnaire remain basically the same with the addition of some indication in section B about the making items 24 and 25 should be made control of teacher hiring policies at the more specific along the focus described school level by the district or LEA. above; right now they are far too general Sections D and E should be changed the to be of much help.If there is space left most. Section D gathers information on after this, here is the place where some several federal programs and one local information on the principal's management program of "choice." This section is a philosophies and approaches should be mini-survey of districts since none of this collected. is tied to the target school.Also, some of these same federal programs are asked The School Questionnaire has four about in the school questionnaire. Does sections.Section A on school NCES need both estimates? Section D characteristics should be kept, but perhaps needs to be redone; it should provide a it could be collected in a less burdensome picture of how district, state, and federal fashion (see CCD suggestion above). programs have an impact on the target Section B is the basic staffing of the school. As it cannot necessarily include school, which is important to keep, even all such programs, a sampling is enough, though it does not tell us much about the but the key is to capture something about direct management of the school's faculty if and how federal, state, and district and staff.Section C is on programs and programs have an impact on the target services.I would suggest that this section school.Section E should be the place be redone and be the section where most where district policies and governance of the way the school organizes students about each of the main school components and their programs is collected. Some of are examined as suggested above.I the programs here are the same ones asked recommend that what is currently in this at the district level and if the district items section E be deleted to make room for new are removed, these will have to stay. My items. problem here is that most of the programs here are entitlement types for special Principal and School Questionnaires student populations which may be a small part of what the "average school" does Most of what I have proposed about a with most of its students. This is the place more organizationally orientated SASS where some of more extensive information falls on these two questionnaires. The on the curriculum and student flows across current Principal Survey asks a number of courses should be collected.Section D is questions about the principal's background. a mixture of items. Item 33a is important I would like to see the general area kept, and is the kind that I recommend in but greatly reduced. Items 1 through 23 general on decision-making.I would give and 26 through 30 should be cut down by up the other items in the section in favor at least two-thirds. I think just some of the of a more systematic view of how the basics about principal background is school organizes its main components. sufficient.In their place new items should be developed that examine how the School Teacher Questionnaire administration of this school manages faculty, students, curriculum, and outside This section of SASS is the hardest to influence on the school. The decision- change, but fortunately not much needs
27 37 change to bring it into line with what I be done to a new SASS. This plan does have proposed. The information collected not include what needs to be done to SASS here provides much of the data for the to incorporate more school level outcomes national profiles of teaching as a such as achievement, promotion, andso profession from a number of perspectives. forth (see point IV above). It has proved very important to NCESas a way to monitor the nation's K-12 faculty in a way not available from any of its How Different in Focus Woulda other surveys. There are nine sections to New Organizational SASS Be from this questionnaire; I would suggest keeping the Current SASS? most of the items in all sections.Sections A, B, G, and H provide the basic dimensions of the teacher and her/his job. SASS has evolved to have three main foci: Section I is a small single item on LEP (1) teacher demand and shortage; (2) and section J is a technical item. Section condition of teachers and teaching as a C is a long section on training. Much of profession; (3) basic organizational this captures information on instructional structure of schools. And the importance support for the teacher from the target of the foci in terms of initial questionnaire construction corresponds to the above school, so it's very useful.Section D is order. What I am suggesting here is on teaching load.If a new SASS included a more school level questions on course reversal of that order. Make the structure, some of this might replace the organizational focus more prominent with a correspondingly larger share of space on teacher level items in this section.Section E is very important and most of the items certain questionnaires; keep the focus on are exactly in line with how teachers teaching and its profession as the second perceive and experience the school as an focus and move the issue of teacher organization. While one could quibble demand and shortage to be the third focus. with some of the items in this section, in the main, it is very useful to an I suggest lowering the emphasis of SASS organizational view of the school from the on teacher demand and shortage for teacher's point of view. several reasons. Although I think that the general area of supply and demand is This example of a way to implement a new useful and there has been some valuable organizational SASS does not mean an work done on this for NCES, it might be extensive overhaul of the full survey. The that too much of SASS is used for this purpose. From past work we now know Teacher/Demand District questionnaire is the most changed followed by the Principal how to estimate basic supply and demand and School questionnaires. Most of the models efficiently from relatively few School Teacher questionnaire should variables, the real question is how detailed remain the same with some change in and nuanced do we need to make the emphasis of some items. Without estimates by adding further variables (Boe constructing the actual new items and re- & Gilford, 1992). Given that the issue of working current items it is hard to tell how supply and demand for teachers has not proved to be the large policy issue that it naive this implementation plan is, but it is an approximation of what would have to was once thought to be, perhaps NCES should only provide the most basic
28 38 estimate of teacher supply and demand. organizations in specific, would suggest For example, does NCES currently need about planning an organizational database estimates of teacher supply and demand on schools. Needless to say, the literature from both a district and school on formal, complex organizations and the perspective.I understand that aggregated parallel one on schools as formal, complex estimates are built up from the unit below organizations is too voluminous to review (Barro, 1992), but if we have good state here. But there are several important estimates constructed from district data lessons from this literature that are helpful would not that be enough to provide in designing a new organizational SASS.' information about the general issue?And if this basic data indicated a large problem Although Formal Organizations Have Far or other related issues arose, perhaps More "Sloppiness" in Their Boundaries supplemental surveys could be used for a Than Originally Assumed, They Are more detailed assessment. Observable and Can Be Studied as Organizations
Where an organization stops and starts is Organizational Theory and the not as easily determined as what was once New SASS thought. Research on all kinds of organizations in both the public and private Perhaps the most useful of NCES surveys sectors shows that formal organizations are those that correspond to strong have a certain fuzziness to their research literatures and theories which in boundaries. A school may be a school, turn have a major impact on educational but it is also part of a district or a set of research and policy.This kind of a private schools. Or a PTA is a part of a connection links a NCES survey to a school but it is also an organization within research field and related policy domains. a school that also has connections outside This is helpful in both the development of of the school and so forth.Nevertheless the survey and in maximizing the influence organizational research has coped with the gathered information has on the these characteristics of formal education establishment. The link between organizations and has shown that, by in assessment and psychometric theory and large, organizations such as schools can be NAEP, and the link between theory on assumed to have enough of a traceable both adolescent development and boundary to make them a suitable entity educational attainment and HS&B/NELS for study. So it is possible to consider are two examples of NCES surveys that organizations such as schools as discrete are strengthened by this kind of a units that can be the focus of a survey. connection. An advantage of shifting the Past SASS's have already shown this to be focus of SASS to a broader organizational operational. The new SASS proposed here one is the ability to have SASS correspond would increase the focus on the school as to the large literature on organizational the main unit of the survey with, as is now theory. the case, supporting data collected from teachers and districts or LEAs. The proposed approach is in line with what research and theory on organizations in general, as well as on schools as
29 39 Organizational Actors Can Provide Reliable work. Observing theprocess of and Accurate Assessments of Collective management and the social organization of Processes that Make Up an Organization schools instead of just the structure of schools leads to a more accurate This is a basic tenet of organizational understanding of schools (Barr & Dreeben, research. Surveying people holdingan 1983; Weick, 1982). Expanding the organizational role is often used as a organizational scope of SASS to examine method of collecting information about organizational processes is consistent with organizations. There are, of course,some the widely accepted "loosely coupled" problems with having individual actors perspective on the way organizations represent whole organizations, but these work. problems turn out to be no greater and not very dissimilar from problems with Organizations Are Less Rational Than They surveying individuals about themselves. Were Once Thought to Be Additionally, surveying actors about organizations has the advantage of This central observation about providing multiple views through multiple organizations suggests that theyare not actors, as has been used in SASS reports infinitely rational, rather organizationsuse on schools and teaching conditions derived a form of "bounded rationality" (Simon, from aggregated information from teachers 1955). This idea is related to point 3 at the same school (e.g., Ingersoll & directly above, but adds to it the notion Bobbitt, 1995). Organizations offer the that decision-making in organizations is the additional advantage of generating other key operation to observe to determine their useable sources of information about basic essence (March & Olsen, 1976). themselves such as administrative records And further, decision-making is alsonot and financial records (e.g., Scheuren, mechanical but a very compromised 1995). process. The point then for SASS is to collect information on how the workings of Technical Processes within Organizations an administrative system in a school makes That Connect an Organization's Goals to Its decisions, exerts control, and how that Output Are Far Less Tightly Coupled Than same administration can ignore other areas Was Once Assumed, Making Informationon Social Organization More Informative Than (Hannaway, 1989). This kind of Information on Organizational Structure information will yield a more accurate description of schools for policy-makers The way many organizations work,or how interested in school reform. they move from goals to means to ends, is best characterized by an image of "loosely coupled" connections (Weick, 1979; Orton Why a Database on Schoolsas & Weick, 1990). This is particularly true Organizations? of schools. The outcomes of achievement and socialization of students are not easily As mentioned, there are four mainreasons connected to many of the processes within why I recommend that SASS be renewed. the school. In other words, a mechanical Let me briefly describe each one. view of schools as organizations is too simplistic to understand how schools really
30 40 Reason 1: What SASS Has Become student achievement and the role that the Department of Education has come to play A recent NCES working paper noted that in education. The federal role in funding "SASS is an unusual education survey" programs for special populations and (Ingersoll, 1995). Meaning perhaps, that undertaking summary evaluations of the reason behind the original design of national achievement give NCES a SASS and its intended statistical powerful motivation for NCES to examine contribution are very different from other what students learn and how are they NCES surveys. SASS was not designed to progressing in school. To some degree, measure the academic progress of K-12 what has been pushed out of the way by students, nor was it designed to measure this strong agenda is information on how the impact of schools on academic schools are organized. learning, nor was it designed to weigh the costs and benefits of federal educational The lack of a survey chiefly dedicated to entitlement programs. SASS was schools as organizations, instead of the originally designed to provide information schooling of students or the achievement on K-12 teachers, with specialemphasis on of students, is a weak point that leads to teacher supply, demand, and quality. But some peculiar holes in NCES's reporting. in the course of doing that, SASS has also For example, as I mentioned above, NCES provided information on schools.It is the should be able to report much more about only NCES data set in which large the K-12 curriculum and how it is portions of the survey were designed implemented in schools.It is a salient gap around the school as a unit of analysis. when NCES can not provide more The teacher sample is representative of information on such a central component teachers, but even here a significant of schooling. A related example is that portion of the information gathered refers NCES lacks information on how schools to the teacher's school. Without maybe react to various policy changes and initially intending to do so, SASS has reforms over time. Federal policymakers evolved in the direction proposed here. are often interested in precisely this question, and increasingly state educational Reason 2: A New SASS Strengthens agencies are too. Without a survey NCES's Overall Statistical Program dedicated to schools, this becomes very difficult to do.(I take up the issue of When I consider the full array of NCES's school reform and a new SASS below.) K-12 statistical program, I am always struck by several weaknesses among what Reason 3: SASS Has Accrued an Impressive is otherwise a strong program. In general, Set of Technical Capabilities about I think that too much of the K-12 program Surveying Schools is focused on student level information. In part this was historically determined by Faced with the task of developing earlier concern over the effects of federal nationally representative samples of the programs on individual students, such asin wide variety of K-12 schools in the the original motivation behind the design country, the SASS project over the past of HS&B to assess federal programs and decade has generated significant and college attendance. But it is also a sophisticated techniques in surveying function of the importance of measuring schools. Representative surveying of any
31 4. population of organizations can be difficult proposed is directly related to school given the complexities of population governance, which is essentially another dynamics of organizations (e.g., Hannan & term for school management (Raywid, Freeman, 1989). Births and deaths of 1991), so not much more needsto be said organizations can be rapid, transformations about this as a reason to undertakea new of organizational boundaries oftenoccur, organizational SASS. Similarly the and these, plus other similar phenomena, importance of SASS's contributionto are not necessarily spread randomly reporting how financialresources are throughout a population of organizations-- turned into classroomresources has been in short a survey design nightmare. A described above (section 3 of study survey of schools faces the same problems. perspectives). So far I have said little For example, significant private sector of about school reform and anew SASS even schooling in the United States, which though it is a prime example of whyan accounts for about one-fourth of all organizational SASS should be done. This secondary schools, offers a particular is explored in the next section. challenge to a survey.Similarly, the local administration of public schools in terms of mergers, births, and deaths makes A New SASS and Information surveying organizations just within the about School Reform public sector difficult.In addition to sampling challenges, developing reliable Even though school reform is central and valid questionnaires for actors within to the SASS planning process, I have waited organizations is also a major undertaking. to discuss it until after establishing the idea As is evident from substantive SASS of an omnibus organizational reports and the project's own technical survey of schools that does not necessarily focus assessments, the SASS team has the ability on any particular reform or restructuring to generate representative samples of trend.I did this for two reasons; first, schools and collect useful organizational because of what I see as the nature of information beyond a survey of teachers or reform in the American system and staffing needs. This is a major capability that should be enhanced further. second, because of what I will recommend It has the NCES do to capture relevant reform potential for NCES equal to that of the information. assessment capabilities accrued from NAEP. NCES should recognize this and build upon it. There is a paradox about school reform in the United States:the country provides a large amount of school reform movements Reason 4: A New SASS Would Bean Important Vehicle to Add to NCES's but the content of these reforms is mostly Capability to Provide Information on School cyclical.Over the past century there has Governance, Finance, and Reform at any one point in time been ample, sustained and serious interest in reform Because these three areas are the focus of schools. And these reforms have had much discussion throughout the American consequences; the development of the age- education establishment, they will remain graded school, the Carnegie unit and the as important topics for NCES for some core academic curriculum, school racial time. Most of what I have already desegregation, and ability groupingare
32 42 some examples of defining educational to capture information about reform reforms (e.g., Mirel, 1994). But while without being tied to any one particular there is the image when the system needs trend over a lengthy time. fixing a suitable reform is fashioned, it is rarely this simple. Reforms at any point An omnibus survey proposed here offers a in time represent political positions about flexible way to collect information about schools that can extend beyond to larger reform without falling into the trap of political orientations.Shifts in the content reform cycles.This kind of a SASS offers of reform are shifts in political power. a reasonable organizational baseline from Deciding the content of educational reform which to undertake supplemental surveys often pits a host of local concerns, of particular reform issues aimed at resources and political orientations against schools.In the year or so after the main often competing educational professions data collection, a Fast Response Survey and loosely linked national business and System (FRSS) or similar means to collect national political parties (Manlove & information on a sub-sample of SASS Baker, 1994). The ups and downs of schools could be used for specific reform political movements have much to do with issues. Then this information can be what is important educational reform at merged with the larger organizational any point in time. database for a powerful set of information. The curricular tracking FRSS, using this This makes it difficult for a statistical design, has shown that this is technically agency like NCES to plan long range feasible, reasonable efficient, tolerable on surveys about particular school reforms. school response burdens, and analytically Although reform in general is a constant profitable. A variation on this idea would topic of deep interest within the American be separate modules of questions about education establishment, it is difficult to specific reform issues given to sub-samples decide what exactly should be addressed within the overall SASS sample during the within an ongoing statistics program. major data collection. Maintaining a basic What is "hot" today is not tomorrow, but survey of school organization at regular it may be back in ten years.If a project intervals with the option to combine this like SASS, which plans to be a long term smaller, focused data collections on series of surveys, is initially too oriented passing issues of concern to the American toward a particular trend in education it education establishment is an efficient and runs the risk of eventually becoming flexible way for NCES to stay current. irrelevant with the rise and fall of specific reform issues. And irrelevancy is the Finally, it should be pointed out that the worse of all possible fates for any current school reform movement happens statistical program. Therefore I would to be heavily oriented towards some of the recommend that no matter how school management issues that I fundamental, how trendy, how earth- recommended a new SASS collect (e.g., shattering any single issue seems for the Elmore, 1992).This, in the short term, is country's educational establishment at any one additional advantage to what is point in time, NCES should not establish proposed here. But given what the history major, long-term surveys around such of educational reform movements in the issues.Rather the key is to think of ways United States shows, I would not predict
33 4`A this particular focus will last. Thecontent Brewer, D., & Stasz, C. (1996). and focus of reform will surely change, Enhancing opportunity to learnmeasures but a flexible combination ofa omnibus in NCES data.In G. Hoachlander (Ed.), survey of school organization and other From data to information: Newdirections supplemental collections will continueto for the National Center for Education be the best way for NCES to providea Statistics (NCES 96-901). Washington, wide range of information about the DC: U.S. Department of Education, nation's elementary and secondary schools National Center for Education Statistics. and any reform issues. Coleman, J., Campbell, C., Hobson,J., McPartland, J., Mood, A., Weinfeld,F. References & York, R. (1966). Equality of educational opportunity. Washington,DC: Baker, D., Alsalam, N., & Smith, T. U.S. Government Printing Office. (1994). Thoughts on a new organizational survey of American schools (internal Elmore, R. (1992). Restructuring memorandum). Washington, DC: U.S. schools: The next generation ofeducation Department of Education, National Center reform. San Francisco, CA: Jossey-Bass for Education Statistics. Publishers.
Baker, D., Ralph, J., & Manlove, J. Hannan, M., & Freeman, J.(1989). (forthcoming). Curricular tracking Organizational ecology. Cambridge, MA: policies and practices in American public Harvard University Press. high schools. Washington, DC: U.S. Department of Education, National Center Hannaway, J.(1989). Managers for Education Statistics. managing: The workings ofan administrative system. New York: Barr, R., & Dreeben, R. (1983). How Oxford University Press. schools work. Chicago: University of Chicago Press. Ingersoll, R. (1995). An agenda for research on teachers and schools: Barro, S.(1992). Models for projecting Revisiting NCES's Schools and Staffing teacher supply, demand, and quality: An Survey (NCES 95-18). Washington, DC: assessment of the state of the art.In E. U.S. Department of Education, National Boe & D. Gilford (Eds.), Teacher supply, Center for Education Statistics. demand and quality. Washington DC: National Academy Press. Isaacs, J., Garet, M., & Sherman, J. (1996).Strategies for collecting finance Boe, E., & Gilford, D. (Eds.).(1992). data from private schools (NCES 96-16). Teacher supply, demand and quality. Washington, DC: U.S. Department of Washington DC: National Academy Education, National Center for Education Press. Statistics.
34 44 Manlove, J., & Baker, D. (1995). Local Weick, K. (1976). Educational constraints on opportunity to learn organizations as loosely coupled systems. mathematics in high school.In M. Administrative Science Quarterly, 21, 1- Hallinan (Ed.), Making schools work: 19. Promising practices and policy (pp. 133- 153). New York: Plenum Press.
March, J., & Olsen, J.(1976). Ambiguity and choice in organizations. Oslo, Norway: Universitetsforlaget
National Center for Education Statistics. (1994). Curricular differentiation in public high schools (NCES 95-360). Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.
Orton, J., & Weick, K. (1990). Loosely coupled systems: a reconceptualization. Academy of Management Review, 15, 203-223.
Ray wid, M. (1992). Rethinking school governance. In R. Elmore, (Ed.), Restructuring schools: The next generation of education reform. San Francisco, CA: Jossey-Bass Publishers.
Scheuren, F.(1995). Administrative record opportunities in education survey research. In G. Hoachlander (Ed.), From data to information: New directions for the National Center for Education Statistics (NCES 96-901). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
Simon, H. (1955). A behavioral model of rational choice.Quarterly Journal of Economics, 69, 99-118.
35 45 1. As is evident from the title, I have taken some liberty withmy assignment to prepare a statement on what the next Schools and Staffing Survey (SASS) shouldinclude in terms of school reform, governance and finances. Although I address each of these, I doso by proposing that the next SASS be made into an omnibus survey about the internal organizationof K-12 schools. The main focus of SASS should be on how schools manage faculty, students,curriculum, and resources; how decisions are made about these; and what kinds of administrative controlsare in place to implement decisions. The objective is to collect information that would provide NCESwith a basic picture of how schools work as organizations. Then, if need be, supplemental informationcould be added to address specific trends in education as they occur. A re-design of SASSpresents NCES with a major opportunity to capture school-level information that will increasingly be of importanceto its statistical program.
I propose reversing the original order of priorities of SASSto one of first, organizational and managerial information, then second, teacher and teaching information,and lastly, teacher supply and shortages. The past SASS's steadily collected more school organizationand managerial information over the course of the last three surveys, so while what I propose would bea renewed SASS, it would not be a radical shift for the SASS project.Additionally I recommend that much of the current SASS remain the same, but with some key modifications to crucial questionnaires.As instructed, I have not written actual new items, but I have tried to describe whatnew items should be constructed, where they should be placed in the survey and what their informational intentmust be.
Also, as instructed, I have written this for the "insider" reader who isfamiliar with NCES and SASS. For those readers who are not "insiders," two publications make goodcompanions to this document: the most recent Programs and Plans of the National Center for EducationStatistics and SASS and PSS Questionnaires, 1993-1994.
Much of what is written here originates from an internal memorandumTom Smith, Nabeel Alsalam, and I prepared while I was an AERA Senior Fellow at NCES (Bakeret al., 1994). Although I assume complete responsibility for any still-half-baked ideas, I owemy two colleagues much for their help in thinking about an organizational survey of schools for NCES. I also would liketo thank Joel Sherman, Tom Parish, and Jay Chambers at AIR for their helpful comments about SASSand finance issues. And thanks to Maryellen Schaub for her comments on an early draft.Lastly, thanks goes to John Mullens at PSA for his kindly monitoring of my progress during work on this document.
I have made a few changes to this draft after the January 25, 1996, presentationto NCES. I would like to thank Susan Fuhrman for her helpful comments. The revisions herewere made in light of the comments in other papers made at the half-dozen sessions that I attended and frommy reading of all the other planning papers.
The ideas and recommendations herein are those solely of the author actingas a private consultant to NCES and do not necessarily represent the positions ofany organization with which the author is affiliated.
2. SASS could prove a vehicle to fill the gap on private school finance, although this isa separate issue from what is recommended here (see Garrett, 1996).
3. Some might ask, why not focus on classrooms? Classroomsare the key point of instruction and are arguably the most basic organizational unit in the formal educationprocess. This is true, but equally true is that inputs made to classrooms and decisions about what occurs thereare made at the school- level (or above). Schools are organizations controlling sets of classrooms.Some information from a subset of a school's classrooms might be useful, but the main focus of SASSshould be kept on the school as the primary organizational unit of survey.
36 46 4. Just a few citations are offered in the following sections as examples on the larger literature on each point; they do not represent a full review.
37 47 TECHNOLOGY FOR K-12 EDUCATION: ASKING THE RIGHT QUESTIONS
Kathleen Fulton, University of Maryland
Overview within schools affects its usage and impact on teaching and learning. Since the introduction of microcomputers To assess this impact, we need to into schools in the early 1980s, there has know not only what equipment is been widespread fascination with the role available, but also how and where technology can play in education. The it is available, to whom it is promise of computers' raises a wide range available and how easily it can be of policy questions: Do they improve accessed. education? Are they worth the cost? Are they being used appropriately? (2) Effective adoption of computers in schools is not a one-time event; it To answer these and other questions, requires ongoing exploration and policy makers and educators need solid experimentation by teachers and data about how technology is used in the students alike. Teacher training is classroom, how it affects teaching and critical to the effective use of learning, and the benefits it provides to classroom technologies, but the students and to teachers. However, much kind of training computer of the available data to date has focused on technologies demand must be the number of computers and related accompanied by continuing equipment in schools. While an important just-in-time support as needed. piece of the puzzle, this data has limited Without considering the stages of value for answering most policy questions adoption of new technologies, and because it gives little insight as to how how that adoption is supported, computers are actually being used by data about computer usage will teachers and students.Similarly, surveys continue to be very difficult to of state policies regarding technology are interpret. not clear predictors of how much, how often, and in what ways various (3) How computers are used in schools technologies are used in K-12 classrooms. is heavily influenced by teachers' conceptions and past experiences. As we conceptualize the data that would For teachers to use technology, adequately address key pedagogical and they must see its value for teaching policy concerns regarding educational and learning in ways that make technology, we should consider several sense to them. Because the important points: definition of what constitutes "best" use of technology continues to (1) The location of computer evolve and change, the process of equipment and other technologies technology adoption has been
39
48 further complicated. Educators cross-sectional and longitudinalsurveys, need opportunities to develop and combinations ofsurvey and targeted visions of how technologies can be comparative case studies, to helpprovide used to support those aspects of important and currently unavailable teaching and learning thatare information needed by educational central to them in their classrooms policymakers and the publicas they seek today, and opportunities togrow in to make sense of the complex relationships facility and comfort with between data on technology andimpacts technologies as they evolve. on student learning.
(4) Computers and other technologies can be adapted to traditional models Good Numbers Are Hardto Find of education, or they can support entirely new ways of teaching and Computer and communication technologies learning. To formulate and pose many opportunities for improving interpret data on the impacts of our education system. To explore andevaluate educational technologies, we must that promise, we need dataon how consider the instructional models computers are currently being used and that underlie their design, on the impacts of that usage. So far,there implementation, and use.If the use have been only a handful of systematic, supports new approaches to representative, consistent data-collection instruction, it is appropriate to efforts relating to theuses of computers in consider measures and outcomes schools.It has proven difficult to interpret that go beyond traditional student the results in order to answer important scores on standardized tests.In pedagogical, budgetary, and policy-related addition to alternative measures of questions. student learning, teacher productivity, empowerment, and The U.S. Congress wasamong the first to professional growth are also seek a better understanding of theuse of important factors that should be technologies in public and private better understood and appreciated. elementary and secondary schools. As early as 1982, the Congressional Office These issues raise important implications of Technology Assessment (OTA) published for NCES in its mission to collect a report on "Information Technology and Its information of value to policymakers. If Impact on American Education". Three educators are to understand the role of years later Congress asked the OTA to technology in schools, they need better provide an overview of theuse of ways to collect, compare, and evaluate computers in Chapter I and for limited data that address these issues. While this English proficient students; this studywas kind of data is not the sort commonly followed by the more comprehensive 1988 collected in the School and Staffing report, Power On! New Tools for Teaching Survey, it may be appropriate for NCES to and Learning, a review ofcomputer usage consider supporting other forms of data and issues in K-12 schools. Other collection, including in-depth studies, educational technology studies followed,
40
49 on distance learning technologies for K-12 on specific applications such as distance education, on technology for adult literacy, learning technologies and programs. and most recently, the April 1995 report Clearly, the market forces require accurate on teachers and technology. data for projections and purchases and for the planning of initiatives. In each of these studies, the first question asked by the Congress, not to mention the However, as the data improve, they will press and the general public, is one that continue to illustrate a fundamental point: NCES could appreciate--"What are the information on the numbers of numbers?" As a member of the OTA staff machines--computers, CD-ROMs, for several of these projects, I soon videodiscs, satellite dishes, learned that exact numbers were difficult telecommunications networks--while to pinpoint. Much of our early data came important, is not the key element. As the from three sources:(1) the 1985 National numbers grow, they are also taking on Survey of Instructional Uses of School new meanings as the technology changes. Computers, conducted by the Center for These changes add new factors that help the Social Organization of Schools at Johns determine the impact that a computer may Hopkins University, under the direction of have in an educational setting: Henry Jay Becker, and summary newsletters from that source; (2) databases The power and capabilities of the from the Curriculum Information Center of computers themselves are changing Market Data Retrieval, Inc.; and (3) constantly. One cannot simply databases from Quality Education Data, compare the numbers of older 8 bit Inc. machines with those that are 32 bit machines--the implications of The data were difficult to compare due to power and speed are of magnitudes different survey and sampling techniques, of difference. However, it is but we reported the numbers as we found encouraging to note that recent them: somewhere between 1.2 to 1.7 surveys (e.g., Quality Education million computers for instructional use in Data, 1996) also report on the K-12 public schools in 1988, growing to numbers of computers capable of almost 5 million computers for running multimedia applications. instructional use in K-12 schools in 1995; 10 states promoting distance learning in Computers are being configured in 1987 but virtually every state using some new ways, with technologies that form of distance learning by the fall of were once stand-alone now being 1989. incorporated into a single unit. We are no longer just counting separate The quality of current data is improving PCs, separate hard drives, because commercial providers have found videodiscs, or projection devices; it valuable to those marketing technology increasingly, the components are to the K-12 sector.Several other firms built in ways that confound counts and consulting groups make it their but increase capabilities. business to collect data on a regular basis
41 Networking further confounds the helpful in providing at leasta basic situation, as one machine can understanding of how various state policies support dozens of students or affect access to technology, but itwas not applications at the same time. able to provide greater insights into the actual use of the technology. Thesurvey The variety of software applications looked at the following questions: in use in the education setting is exploding; the nature and power of (1) Does the state require or the applications is as significantas recommend that public schools the numbers of machines integrate computer or information themselves. technology in the curriculum? (All but six states did.) Thus, unlike the relatively constant numbers collected on teachers and (2) Does the state require public administrators in the SASS, and the well schools to offer computer-related understood assumptions about what these courses such as keyboarding or numbers mean, numbers of pieces of computer literacy for students? technology are subject to far more (Twelve reported the requirement complex interpretations. for students.)
(3) Does the state have a mandate for But Even Numbers Are Not computer competency or Enough performance standards for students related to information technology? From a policy perspective, however, it (Twenty reported mandating student was the information we could not find that computer competency.) became the most intriguing, and the most educationally significant, piece of the (4) Does teacher certification in the puzzle. In the Teachers and Technology state include a requirement for report, for example, there were many pre-service computer or technology questions Congress asked OTA to training? (Nineteen states said yes.) consider. How do state policies affect access to technology in schools? How do (5) Does the state have a requirement teachers use the technology that is for inservice computer or available to them? What impacts do the technology training? (Only two technologies have? We contracted with states reported such a requirement.) experts in educational technology data analysis to help us find answers tosome of To assess whether the state requirements these questions. Ronald Anderson of the were in any way related to the numbers of University of Minnesota conducteda computers in classrooms, Anderson looked review of state technology activities related at the survey data in relation to the state to teachers. This investigation, a microdensity data reported by QED's 1994 telephone survey and review of state report on Technology in Public Schools. planning documents and guidelines,was We were surprised and frustrated to find
42
51 that greater state technology requirements technology presence and use is simply not did not necessarily mean more computers available". in the classroom (or vice versa). Anderson found a low of 8.1 students per Becker found the best data in the 1992 computer in Wyoming (a state that only Computers in Education Study of the IEA requires one of these factors (computer (International Association for the training for teacher certification); to a high Evaluation of Educational Achievement). of 22 students per computer in New Although their sample of schools was Hampshire, a state in which three of the small (571 schools, with responses from five policies are in effect (promoting computer coordinators) the IEA study technology integration in the curriculum, asked questions about utilization, processes requiring computer course for students, of decision making, and attitudes. and requiring computer training for teacher Students reported their own computer certification).This leads us to suggest that experiences.Interestingly, the time the relative amount of computer reported for computer use was much technology in a state should be used with higher when reported by computer great caution as an indicator of that state's coordinators (who reported that computers commitment to technology in instruction. are used about 1 3/4 hours per student per week at the elementary level; two hours per student per week at middle schools, What Do We Know about How and approximately 3 hours per student per Computers Are Used in Schools? week at high schools) than when students themselves gave the estimates of computer The compelling questions still evade us: usage (students reported using computers how much are educational technologies 24 minutes per week in grade 5; 38 used, for what applications, by what kinds minutes per week in grade 8; and 61 of teachers, under what conditions, and minutes per week in grade 11). These with what results? For the OTA study, we discrepancies point out the need for greater drew from a number of sources: case sophistication in data collection methods. studies and site visits, conversations with teachers and administrators on site, by Regardless of the time spent using telephone, and at conferences; and computers, the data suggested that the traditional research reviews and Internet kinds of use are traditional.Indeed, the searches. We also commissioned a most common activities on computers for contractor report that reviewed existing elementary students were drills in basic surveys to see how their results might shed skills and instructional games. Also light on the questions of how much, how popular at all levels were general computer often, and in what ways teachers use literacy activities and word-processing. In technology. This was particularly secondary schools, the data suggested that challenging; as the author, Henry Becker computers are used relatively infrequently of the University of California at Irvine, for teaching and learning in traditional noted, "Unfortunately, much of the data academic subjects, far less than in classes needed for a complete picture of focused on teaching students about computers.It would be very useful to
43 have more current data in this area to increasing teacher productivity and assess curricular impacts. professional development.
NCES data are beginning to look at the 1.Location of Technology Affects Use role of technology within subjectareas, as in the 1994 report, How School The location of technology withina school Mathematics Functions and in the data has a significant impacton use.It seems a compendium for the NAEP 1992 basic point, but it stillcomes as a surprise mathematics assessment of the nation and to many educators: when computersare the states. One measure of barrierscan be placed outside the classroom, it is less seen in teachers' responses to the question likely that they will be a part of the regarding confidence in their preparation regular instructional day.In the past, to teach: while 85 percent of grade 4 schools typically aggregatedcomputers in students and 93 percent of grade 8 students labs, in part to assure equal (if limited) had teachers who reported being "very access for all students and teachers. But well prepared" in mathematics concepts, lab time is time that must be scheduled, only 15 percent of grade 4 students and 21 and it can create logistical nightmares that percent of grade 8 students had teachers discourage use. Furthermore, a trip toa who reported that they felt very well lab can be a "mini-field trip" requiring prepared in computers. This kind of data time to get to the lab, time to get settled, should provide a red warning flag to and time to close down and return to the policymakers if they expect to see greater "real class" on the other end, all eating computer usage in curricular applications. into valuable instructional time. Nonetheless, the Becker study reported that most schools place a majority of their Four Issues Affecting the Use of computers in computer labs--70 percent of Educational Technology all middle and junior highs hada majority of their computers in computer labs in While much of the information cited above 1992. Only 35 percent of all school is limited in scope, difficult tocompare, computers were in the classroom. Some and often anecdotal, its educational schools are finding a way to offermore significance warrants further analysis. flexibility through use of laptop Despite the admitted limitations of this computers, kept recharged and placed on data, there are clear implications from this carts that can be rolled from room to information that suggest issues thatmay room, moving the technology to the provide a better basis for understanding students rather than vice versa. But again, technology use in the classroom. These necessary scheduling and coordination issues include: the placement of technology create barriers to spontaneous or in the school, the kinds of training and continuous usage.It's a bit like having to support teachers receive, and the value of share books, or schedule the use of technology for teachers in terms of the pencils, if you are a regular computer instructional goals and models the user. technology supports, as well as its role in
44
53 This same difficulty lies with However, other factors are also at play in telecommunications networks. Although these instances, notably substantial teacher the recent NCES study for the Department training and support, new approaches to of Education reported substantial growth in curriculum that support technology use, the number of schools that have Internet and greater flexibility of overall class access (growing from 35 percent to 50 organization and scheduling.It may be percent of all schools in one year), the difficult to separate out the location factor, number of instructional rooms with that but it clearly plays a role that should be access, while also improving, still remains considered. low (increasing from 3 percent to 9 percent in the same period.) When one 2.Training for Technology Requires New realizes that "instructional rooms" includes Approaches and New Definitions libraries and media centers, it becomes clear that classroom access is even more Although most teachers have had some limited. However, in considering training about technology, far fewer have deployment models for telecommunications had suitable training to prepare them to systems in schools, cost factors are likely use it in their teaching. For teachers to to lead policymakers to be tempted to use technology effectively, they have to continue to opt for the lab model. The see the value it has for them. Technology recently released "Kickstart" report use has to fit with their teaching style, and estimates that it could cost $11 billion for it has to work with the specific content and the one-time purchase and installation costs skills they guide their students to use. for a lab model (single room with 25 Furthermore, effective use must be based computers, an Ethernet LAN in the lab, on visions of valuable applications, and be and 10 telephone lines); $22 billion for supported by training to develop skill and this configuration plus one computer and comfort with these applications.It also modem for each teacher; and $47 billion takes time and experimentation to adapt for a classroom model in which all the technology into one's routine. Only classrooms have one computer for every over a number of years do most teachers five students, an Ethernet LAN across and become "fearless" with technology. within all classrooms, and a T-1 connection. Timing. Timing of training is particularly important: when training is provided How much more are computers used when before the hardware or software arrives, or they are located in the classroom and are before teachers know what equipment they present in sufficient numbers for regular will be using, there is little opportunity to usage by a good portion of students? practice and apply that training back in the More research is needed to answer this classroom. Classes are too often general, question; however, data collected by high not focused on the level of expertise of the intensity programs like the Apple individual teacher or the teacher's needs. Classrooms of Tomorrow, the Buddy It is ironic that we repeat the same Project in Indiana, and the Christopher mistakes in professional development as in Columbus School in Union City, New the classroom: lecturing as if all learners Jersey, suggest that usage is much higher. are starting from the same point, rather
45
S4 than realizing that teachers have unique Education in 1992, offer guidelines for the experiences, understandings and needs accreditation of educational computing and which they bring to the class. technology programs. These suggest that prospective teachers need to demonstrate Technical support. Finally, technical knowledge and use of computers fora support is crucial: without assistance and number of teaching activities. trouble shooting for the inevitable problems that occur, teachers may be The ultimate criteria is, however, the overwhelmed by frustration that can negate marketplace; if districts and states willnot all their prior positive enthusiasm and hire new teachers who cannot demonstrate desire to apply new technological skills. proficiency with technology in their fields, or, as is the case in Michigan, require Teacher preparation. We also need better that teacher candidates demonstrate understanding of how new teachers are knowledge of computer applications to the prepared to teach with technology. satisfaction of the school or district before Finding little data in this area, OTA that individual may engage in student contracted for a review on this topic for teaching, education schools willno longer the Teachers and Technology report. be able to avoid this responsibility. To Although this was a limited survey with date, however, only a limited number of some methodological problems, it states or districts have made this a reinforced the message heard in case requirement. studies, interviews, and conversations with new teachers: technology is not central to As noted above, the Anderson study the teacher preparation experience in most indicated that fewer than half the states colleges of education. Although most (19) require training in computersor teacher education students take some form technology for all teachers seeking of computer literacy course, they typically certification.Furthermore, this can vary graduate without a clear vision of theways from a semester course to a demonstration technology can be used in their of the use of technology in teaching. professional practice.Clearly this is an More detailed data in this area would have area of concern, especially when a major impact on policy related to the considering that 3.3 million teachers will preparation of new teachers.Instead of be needed for K-12 schools by the year just asking, "How much trainingare they 2003, 1.4 million more than are in the getting?" we should also be asking, "What profession today. kind of training is it, under what circumstances, and with what kinds of It is important to follow trends that affect support?" teacher preparation, specifically state certification standards, and the 3.Technology Should Be of Value to accreditation for institutions of teacher Teachers education. For example, the National Council for Accreditation of Teacher What has most often been missing in Education Guidelines, developed by the discussions of technology for education is International Society for Technology in a view of technology use that empowers
46 55 teachers by giving them reasons to use the productivity (spending the same amount of technology to accomplish valued tasks. time in class preparation and Few teachers have been encouraged to administration, but accomplishing more); view new technologies as professional professionalism (taking on the role of tools that can help them do their jobs experts in certain areas, and sharing their better, more efficiently, or in new ways. successes with colleagues); and Very rarely are teachers asked why they empowerment (pride in the school and want to use the technology, much less their personal proficiency and given models that could guide this vision. accomplishments in an area that is valued Although most teachers believe in the in society). While job satisfaction and value of students learning about computers professional growth are difficult to and other technologies, many teachers are quantify, they are areas of importance to not aware of the resources technology can staffing issues related to teacher turnover provide to them personally, as and continuing expertise. professionals, in carrying out the many aspects of their jobs. Indeed, one of the most powerful findings of the OTA study was the importance of Our lack of attention to teachers' needs is technology for teachers for a variety of apparent in our almost total lack of data reasons. These include enhancing about the types of technology available to instruction that supports new models of teachers.Little data have been collected at learning, simplifying administrative tasks, the national, state, or local level on the and fostering professional growth.I will numbers of teachers (not just classrooms discuss each of these briefly. or students) who have a computer of their own at their desk, or a telephone, or a link 4. Enhancing Instruction: New Ways of to others via local area or wide area Teaching networks, or to the outside world via Internet access. Why is this figure not as Teachers are becoming aware of the important as the overall number of growing body of evidence on the value of computers or telecommunications links technology for student achievement as within a school?I suggest that the fact measured by test scores, as well as its that this question is not asked from the positive effects on student attitudes toward teacher's perspective speaks volumes about learning on students' self confidence. the way teachers are treated as Nevertheless, more contextualized research professionals within the educational on the broad variety of educational effects establishment. of technology is necessary, in order to gain a clearer picture of the value added to Data from one study in Indiana, "A instruction when technology is used. What Computer for Every Teacher," give one is needed is research that gives a clearer window on possible outcomes. In this picture of when, why, and how project participating teachers were given a technologies impact student learning. computer and printer for their use at home or school, along with training and We also need richer studies and better software. Teachers reported greater tools for measuring growth in skills
47 56 important for the information age ifwe are inquiry, authentic challenging tasks, to develop better understanding of student collaborative learning, and growth in inquiry learning skills; in multidisciplinary curriculum, it will be independent problem solving and important to understand if theuse of collaboration skills; in written, spoken, technology does indeed support these and multimedia communication skills; and goals, and if so, how and why? the ability to find, manage, and evaluate information from a number of sources. For example, technology offers valuable Other questions are equally important: assistance to teachers struggling with How can we evaluate students' confidence demands created by performance in themselves as learners? Can we get assessment. Keeping track of rich but better measures of students' appreciation extensive histories of student performance of the outlook and traditions of other over time can be a real challenge. cultures or their development of moral Teachers are finding ways touse video to values and the ability to empathize with record performance, personal digital others? How do we measure students' assistants to keep track of their love of learning and developing "habits of observations of student activities, and the mind?" electronic portfolios to collect and maintain student work on disk. These There is also limited--but growing- technologies allow records to be retrieved evidence suggesting that technology use and updated to demonstrate cumulative can create positive changes in teaching student progress. Technological style. One important study looked at how applications like these make it easier for "accomplished" computer-using teachers teachers to adapt to some of the new believe their teaching changed as they student-centered assessment approaches became more comfortable using that educational reformers are requiring. computers. Teachers in this study reported that they spend more time with Much is needed in the form of additional individual students, expect more from surveys and in-depth case studies and them, are able to present more complex analyses to develop a better understanding material, are better able to tailor of the dynamics of technology's part in instruction to individual student needs, are changing teaching style in these and other more comfortable with allowing students to areas. work independently and in small group activities, and spend less time lecturing or Tools for greater teacher productivity. One practicing or reviewing material with the of the most obvious benefits of computers whole class. We hear of computers for teachers is the streamlining of the making the teacher more the "guide on the myriad administrative tasks that take up so side" rather than "sage on the stage", much of teachers' time. Attendance more a coach or facilitator than records, grades, book lists, classroom all-knowing expert, but we need solid inventories--these and hundreds of other studies to confirm this realignment of time-consuming tasks can be streamlined teaching style. As school reform literature or automated with a personal computer. calls for greater attention to student To use the tools effectively, however,
48
57 teachers must be given the release time to can serve teachers in meeting these new learn the appropriate software in order to challenges.Certainly the new curricular set up and maintain files. standards are going to have a major impact on the training needs of teachers.Just as Teachers also report they can access more the subject matter content and pedagogy resources for preparing and updating recommended by the National Council of lessons through telecommunications Teachers of Mathematics has created a networks, using materials that are more huge need for helping teachers learn to current and more compelling than what is adapt their teaching according to these found in standard textbooks. They can standards, similar national standards in also add their own materials to share with science, history, and English will create others. Curriculum development is a training demands in these curricular areas. tedious process; however, using And, as cross-curricular learning is collaborative software teachers can work emphasized, teachers will have to become together in creating new materials, more comfortable with content and drawing from the resources each can activities outside their specialty field. provide. Data from sources like the Eisenhower Clearinghouse electronic Education can take lessons from business catalogue of instructional plans written by in this regard. Industry has found it much teachers in mathematics and science can less expensive, and more effective, to use give an idea of the popularity of this a "training on demand" model, in which activity among teachers. education and information is brought directly to the worker at his or her Perhaps one of technology's greatest workstation. For example, by using an selling points is the way it offers interactive satellite network rather than opportunities for greater communication bringing a "dog and pony show" of with parents, whether the technology is a courses to 12 different cities for 4-5 weeks simple phone in the classroom supplied per quarter, Hewlett Packard has cut its with a voicemail account, or more sales training costs from $2 million to sophisticated school/home $200,000 per year. The sessions now take telecommunications links that allow two days, require no travel, and provide exchange of e-mail messages. With training the employee can use research showing parental involvement as immediately, to improve retention, recall, one of the greatest indicators of student and application. success in schools, any tool that helps teachers encourage that involvement also Applications like this could change becomes a productivity tool for the in-service training for teachers and greatly teacher. enhance options and flexibility. For example, video tapes of teaching based on Tools to support teachers' learning. As NCTM math standards form the basis for states encourage new forms of instruction, the PBS-sponsored Math line series; the changes in content, and new ways to tapes are shown over the local PBS station measure success, it is important to develop and can be reviewed as needed by a better understanding of how technologies participants as they study the concepts and
49 content within them. They are classic cost/benefit models, they should be supplemented by online group discussions set in terms that resonate with of the tapes in which teachers share their policymakers. Do teachers use their time experiences in trying the approaches and in ways that are more efficient? Do exercises back in the classroom, reflecting students have more time for learning, and on what works, what doesn't, and why in is the kind of learning thatgoes on a supportive, collaborative forum. important learning? Can expenditures in other areas that consume large portions of This blend of formal training and informal education budgets--such as materials and support could be the professional textbooks -be cut back as technology development model of the future, giving expenditures increase? Are staffing costs more individualized, just-in-time training affected when technology resources when and where teachers need it.With substitute distance teachers for some teachers adding their own content and courses, when more teacher development experiences, they become supporters and is done informally rather than in formal mentors to their colleagues near and far. courses, or when support is provided by The informal support of peers, as well as student technology aides or local the access to resources and experts businesses models like the US Tech. anywhere in the world, is likely to have a Corps? Are the continuing costs of profound effect on teachers and teaching, technology expenditures (for hardware, not to mention student learning. software, networking, training, and support) worth the drain on otherwise Although the benefits seem obvious, there strapped budgets? has also been little research on how networking is used by teachers, the value While these questions are beyond the of the informal training it provides, the scope of the SASS, they are areas of cost savings it offers, or how much policy concern at all levels and should be teachers value this form of support. Does the basis for comprehensive surveys, fast teacher collaboration increase? In what response surveys, and longitudinal studies ways is this of value to teachers? How that can inform these decisions. As one does this affect their image of themselves reviewer suggested, "A useful plan for the as teachers, as learners, as professionals; School and Staffing Survey would be to and what impact does this have on link the descriptive data collected in that motivation and aspirations? survey with simultaneously funded research that more directly addresses the causal factors that account for variation in Final Recommendations for NCES teacher practices and student outcomes. This coordinated two-pronged approach As noted earlier in this paper, the will help us to understand how to realize expenditures in technology at the K-12 the potential of educational technology to level, some $2.4 billion a year, warrant assist in improving teaching practice." policymakers taking a closer look at how much and how well that technology is One can be encouraged by the other papers used. While these measures may not fit in this series, suggesting how the ability to
so 599 collect and share data electronically make International Society for the Accreditation more complex surveys perhaps more of Technology in Education.(1992). feasible today.I hope that the planning Report from the Accreditation Committee. done by NCES for future studies will take Eugene, OR: Author. these important issues into account and that valuable NCES resources can be Means, B., & Olson, K. (1995). leveraged to the greater benefit of Technology's role in education reform: educational improvement. Findings from a national study of innovative schools. Washington, DC: U.S. Department of Education, Office of References Educational Research and Improvement.
Anderson, R. E.(1994).State National Center for Education Statistics. technology activities related to teachers (1993). Data compendium for the NAEP (NTIS Report No. 95-184800). 1992 mathematics assessment of the nation Springfield, VA: U.S. Department of and the states (NCES 23-ST04). Commerce, National Technical Washington, DC: U.S. Department of Information Service. Education, Office of Educational Research and Improvement. Becker, H. J.(1994). Analysis and trends of school use of new information National Center for Education Statistics. technologies (NTIS Report No. 95- (1994). How school mathematics 170981).Springfield, VA: U.S. functions (NCES 23-FR-02). Washington, Department of Commerce, National DC: U.S. Department of Education, Technical Information Service. Office of Educational Research and Improvement. Becker, H. J.(1996). Comments on January 1996 draft of "Technology for K- Reinhardt, A. (1995, March). New ways 12 education: Asking the right questions". to learn.Byte, 66. Unpublished document, University of California, Irvine. Sheingold, K., & Hadley, M. (1990). Accomplished teachers:Integrating Heaviside, S., Farris, E., Malitz, G., & computers into classroom practice. New Carpenter, J.(1996). Advanced York: Center for Technology in telecommunications in U.S. public Education, Bank Street College of elementary and secondary school, 1995 Education. (NCES 96-854). Washington, DC: U.S. Department of Education, National Center Software Publishers Association.(1995). for Education Statistics. Report on the effectiveness of technology in schools, 1995-96. Washington, DC: Author.
51 60 U.S. Advisory Council on the National U.S. Congress, Office of Technology Information Infrastructure.(1996, Assessment. (1995, April). Teachers and January). Kick Start Initiative:Connecting technology: Making the connection (OTA- America's communities to the information EHR-616). Washington, DC: U.S. superhighway. Washington, DC: Author. Government Printing Office.
U.S. Congress, Office of Technology Willis, J., Austin, L., & Willis, D. A. Assessment. (1982, November). (1994). Information technology in teacher Informational technology and its impact on education: Surveys of the current status American education (OTA-CIT-187). (NTIS No. 95-170999). Springfield, VA: Washington, DC: U.S. Government U.S. Department of Commerce, National Printing Office. Technical Information Service.
U.S. Congress, Office of Technology Assessment. (1987, March). Trends and status of computers in schools: Use in Chapter 1 programs and use with limited English proficient students.Staff paper.
U.S. Congress, Office of Technology Assessment. (1988, September). Power on! New tools for teaching and learning (OTA-SET-379). Washington, DC: U.S. Government Printing Office.
U.S. Congress, Office of Technology Assessment. (1989, November). Linking for learning: A new course for education (OTA-SET-430). Washington, DC: U.S. Government Printing Office.
U.S. Congress, Office of Technology Assessment. (1993, July). Adult literacy and new technologies: Tools for a lifetime (OTA-SET-550). Washington, DC: U.S. Government Printing Office.
1. Note: For the purposes of this paper, the term "computers" is often used interchangeably with the word "technologies," since many of the technologies used in schools are, in fact, computer-based (e.g., CD-ROMs, printers, disk drives, personal digital assistants and other hand-held devices, and networks offering telecommunications linkages).
52
61 LINKING STUDENT DATA TO SASS: WHY, WHEN, HOW
Phillip Kaufman, MPR Associates, Inc.
This paper considers the feasibility of new longitudinal study of high school linking a student data sample with the students.If it is not feasible to collect all SASS teacher and administrative data. of these data as they have been collected in NCES has from time to time considered the past (regardless of their analytical linking their student-based elementary and merit), it seems to me that ways must be secondary surveys to the school- and found to collect all of these data with new teacher-based surveys. These thoughts methods or systems. Unless we think hard have usually been centered on the about these issues, opportunities may be analytical power that such a lost and we will create gaps in our student/teacher data set would hold. knowledge about American schools and the Budgetary concernsin terms of both impact of the last few decades of reform. fiscal program budgets and burden Old ways of conducting the business of budgetshave also been important, but the data collection may have to be modified in analytical justification of linking student light of the increased budgetary constraints data to teacher data has generally taken imposed by Congress and the President precedence. However, speculation on the and the simultaneous increase in analytical feasibility of linking datasets is of demands of the public. particular importance now in the current climate of budgetary constraints and distrust of federal data collection among Some Ground Rules segments of the public. In this paper I discuss the rationale for NCES (and indeed all of the governmental linking a student data collection in SASS statistics community) is entering an era and then explore several options for when hard choices need to be made collecting these data. However, before concerning data collection and reporting. launching into the main body of the paper, In an era when we face increasing I would like to lay out a few principles to demands for more and better data from a organize my discussion of these issues. wide variety of educational policymakers These guidelines have to do with (1) what and researchers, we are also being asked dimensions of student data should be to do more with fewer resources in terms collected, and (2) what criteria should be of both program money and personnel. used to judge the merit of the proposed Thus it may no longer be feasible to new data collection system. collect data on schools, teachers, and programs through the Common Core of Dimensions of student data. As I will try Data and the Schools and Staffing Survey, and develop further in this paper, it seems conduct another National Assessment, to me that at least three aspects of students continue to track the early childhood are important to track and should be a part cohort of students (in ECLS), and launch a of any system of student data. While
53
62 perhaps not necessary elements of a SASS that for such a merger to makesense, it student component, these elements should must pass at least one of two tests.First, be (and are) part of the overall data it must make sense in terms of efficiencies strategy for NCES and should be of administration or respondent burden. considered when thinking about what kind That is, it must prove to save budgetary of student data should be attached to resources (either program budget or SASS. These elements are: overall burden budget). Secondly, it should make academic performance, growth in analytical sense. That is, themerger achievement, and successful transition into should result in a data system that allows and through the increasing demands of more comprehensive and sensible schooling and work. inferences to be drawn.
Among these three, measuring overall To summarize, I start this discussion academic achievement is perhaps most considering three main elements of student important (for why else can we justify data and two principles of an adequate public and private investment in schools) argument for linking student data within and has traditionally had the most SASS: attention. Both NAEP and the longitudinal studies have made estimates of overall Elements of a student data ,system: achievement levels of various groups of students over time.Accurately measuring Overall academic performance, growth in achievement (especially in Growth in achievement, and observational/survey data) is perhaps the Successful transition through the most challenging. This has been done by increasing demands of schooling analysts using both HS&B and NELS:88. and work Examining critical transitions has historically received the least attention, but Criteria for judging the adequacy of has come under increasing scrutiny as the including a student component in SASS: educational community has realized the importance of studying the life-course and The merger should produce some its impact on education (Pallas, 1993, p. cost benefit, and 20).It seems to me that three main The merger should engender an transition periods are important to keep in analytical payoff mind while considering student data within SASS: (1) the passage from middle school All of the above must also be considered to high school, (2) the path through high in the context of the mission of NCES (as school to graduation, and (3) the transition I paraphrase it): (1) providing to the public from graduation to school or work. accurate information on the "Condition of Education," (2) producing policy-relevant Evaluative criteria. While keeping these but policy-neutral research reports on three elements of student data in mind, I current and/or enduring issues in need to set up a few criteria to judge the educational policy, and (3) providing worthiness of any proposal to linking databases that other analysts can use as individual data with SASS. It seems tome research tools in their own policy work.
54
63 The data needs differ for these three individual student achievement, growth, functionsranging from fairly descriptive and experiences. data for function 1 to data for function 3 with the potential for multivariate analysis It is possible to attach to SASS additional and "cautious causal" analysis. aggregate measures of student characteristics and outcomes. In fact, this is the approach advocated by Don Why Collect Individual Student McLaughlin in his response to an earlier Data through SASS? version of this paper. McLaughlin makes the case for aggregate data based on the Much of the data that NCES collects are tremendous improvements in the not on individual students, but are on assessment practices of many state characteristics of schools and other departments of education. He advocates educational institutions. For example, using these state assessment data SASS now collects data on school districts, (presumably available for each school in schools, principals, and individual SASS) by linking them to the state NAEP teachers.Detailed information is available assessment. Dave Thissen has conducted on characteristics of the curriculum, such equating for the North Carolina state qualifications of teachers, school and assessment. district level policies and practices. Traditionally, what student data have I appreciate McLaughlin's contribution to existed in SASS were generally aggregated this discussion and his comments on using to the school level before being captured. state assessments are well reasoned. For example, percent of students receiving However, as he acknowledges, cross- free lunch, percent of students of various sectional data on student outcomes are less racial-ethnic groups, etc., have been interesting than longitudinal data (and, in attached to the school files since the first my opinion may not be worth the effort of cycle of SASS. collecting at all).Collecting longitudinal aggregate data on student achievement However, while it is important to be able within schools is of more interest, but to accurately measure and track schools, (again in my opinion) not as useful as teachers, and curriculum practices, these collecting individual student data. That is, data provide the context for measuring the aggregate test scores or mean outcomes do main component of educationstudent not capture the individual variation in achievement, growth, and progress. As achievement that traditionally has been of the nation tries to assess and track the such interest. implementation of school reform, the data on schools and teachers do provide For example, the variance of test scores valuable indicators of the extent of within schools has been used as an reformand these data have been used outcome measure in assessing the effectively over the last decade for this effectiveness of schools. High mean test purpose. However, these data are much scores may be due to the school's efforts more policy relevant when used in the at increasing the learning of students context of seeing how they are related to already achieving at a high level or may be
55
64 due to the school's successful attempts at thinks is the status of the student, but that raising the scores of students at the bottom student may have enrolled in another of the ability ladder. Mean scores mask school (perhaps an alternative school) or these important differences in the impact may have taken the GED and received an of school policies and practices. alternative credential.Student data would Furthermore, the transitional experiences also be cross-sectional and vulnerable to and out-of-school experiences are so all of the weaknesses of cross-sectional important to those educational outcomes. data. Therefore, while it may be worthwhile for SASS to explore the possibility of Thus, attaching only individual attaching aggregate and longitudinal administrative student statistics to school assessment data to their regular data and teacher data would miss invaluable collection, it seems to me that still insights that are derived from observing accurate estimates of the associations of student outcomes and transitions in the teacher and school characteristics on context of student's prior experiences, student outcomes necessitates the linking in aptitudes, and ability levels in schooldata some way of individual students (or that can only be measured through similar groups of students) with individual individual student surveys. Administrative schools, teachers, or policies (or similar data also would fail to capture or measure groups thereof). the impact of the transitions that students make through different schools and Of course, an expansion of the current classrooms to the world of work and SASS student survey (based on family life.Clearly, while collecting administrative records of students of student data through administrative records sampled teachers) could add immeasurably may be cost effective, they do not provide to the analytical power of SASS. This the kind of data that add as much to the option would build on the current efforts analytical power of SASSonly individual to include student data in SASS. While student data can do this. current student samples would have to be increased to be representative of the Over the years, NCES has relied primarily school, it still seems reasonable that this on two vehicles for collecting data on would be the most cost effective choice. individual studentsthe National However, it is perhaps the least effective Assessment of Educational Progress analytically.Only limited kinds of data (NAEP) and the system of longitudinal could be collected by administrative studies including the National Longitudinal recordsrace-ethnicity, sex, absences, Study of 1972 (NLS:72), the High School maybe grades. Test score data that would and Beyond study (HS&B), and the be comparable across schools would not be National Education Longitudinal Study of available. Furthermore, while data on 1988 (NELS:88). As an integral part of dropout status may be available from these data collections, individual student administrative records, we have known for data have been directly linked to data a long time that these data are unreliable about the student's teacher, classroom, and as indicators of student status. They may school. NAEP and the longitudinal studies be reliable indicators of what that school accomplish this by including school and
56
65 teacher questionnaires along with student minimizing the burden to individual background and assessment data. Data on students. For example, while no student student outcomes can therefore be linked takes all test items, the 1992 NAEP with data on educational context. mathematics assessment contained 178 items at grade 4, 205 items at grade 8, and However, much of the school, teacher, 201 items at grade 12.This allows and classroom data collected by the reliable estimates across five content areas student-based surveys are collected in in mathematics as well as three ability more breadth and depth in SASS, or in areas. (The mathematics assessment in any case is redundant with data collected NELS:88 in contrast, contains only 40 by SASS. Furthermore, SASS collects items and five proficiency levels.) data about schools, teachers, and, most importantly, school districts that are not NAEP also includes a student collected by NAEP or the typical questionnaire that solicits background longitudinal study.In a time of tight information on each student. NAEP is budgets (that may become even leaner) a built to obtain good estimates of reasonable question is why not borrow the proficiencies in a variety of areas for strengths of both types of surveys and link groups of students. One of the primary the more detailed student data NAEP or a strengths of NAEP is its ability to track NELS to the richer teacher, school, and the overall achievement levels of U.S. district level data in SASS? In this manner students over decades of time. From the each may provide contextual data to better early 1970s NAEP has reported on the interpret the other and possibly reduce the mathematics and reading achievement of overall respondent burden (although elementary, middle school, and high perhaps increasing the burden on those school students. This has provided sampled)thus fulfilling the requirement I educational policymakers and the general set for myself in the introductory section public with an immeasurably valuable tool of this paper. This is the topic to which I in monitoring the health of our educational will turn next. system.
Weaknesses of a Linkage with SASS Linking NAEP to SASS While NAEP has some obvious strengths Advantages of a Linkage with SASS as a candidate for merger with SASS, it also has several weaknesses. Those NAEP has several distinct advantages over aspects of NAEP that do not lend a NELS in such a linkage. The primary themselves to a merger with SASS are advantage is in the content detail that is analytical more than procedural. For provided in the assessment and the age or example, the main weakness of NAEP is grade coverage available in NAEP. Due that it is not longitudinal. Merging a to an adaptation of matrix sampling called cross-sectional SASS and a cross-sectional balanced incomplete block (BIB) spiraling, NAEP would still result in a cross- the design of NAEP allows for broad sectional survey. While the cross-sectional coverage of curriculum content while design of NAEP allows for rich data for
57
uuVIC° descriptive indicator work, the merged As mentioned above, one of the major dataset with its rich contextual data and contributions of NAEP is the trend data assessment data would still be of little use that it provides on student achievement in in producing valid analysis of the the United States.This strength of NAEP, association of school policies and however, proves to be one of the greatest practices. In fact, the existence of such a arguments for not linking it to SASS.It dataset may actually encourage "invalid seems unreasonable to expect that such a but potentially influential studies of schools linkage could be done withoutsome effects that could seriously distort modification of the design of policy."' That is, secondary analysts (or, NAEPeither in its sampling designor its with due apologies to William Raspberry, administration design. Such changes in the a columnist looking at published NAEP design of NAEP could result in changes in reports) could make erroneous conclusions the estimated proficiency levels in the about school policy based on the real but United States.' In addition, SASS isa misleading associations in the data. fairly new and dynamic dataset.Again, given the importance of the NAEP time Another analytical weakness of NAEP is series, one would want to be very cautious that it does not contain good measures of in any changes to the design of SASS that student socioeconomic status' (and may would effect the design of NAEP, in either never contain such measures). Without a content or sampling design. Therefore, measure of this kind, it is difficult to locking the design (and administration) of accurately describe the contribution of SASS to NAEP would make future school process and policy variables on changes in SASS very difficult.For student outcomes. Most of these process example, currently NCES data collections variables are related to student poorly measure the classroom experiences socioeconomic status and/or student body of students. That is, while being able to socioeconomic status.Again, invalid but describe educational inputsstudents, persuasive inferences could be drawn from teachers, schools--they do not measure these data. educational processes wellwhat actually goes on inside the classroom. There However, while socioeconomic status is a would be many issues in incorporating a prominent gap in the student background sample of classrooms within the design of variables provided by NAEP, it is only SASSincluding preserving the trend data one of several variables that one would of schools and teachers from earlier rounds want to collect and measure in order to of SASS. Adding the encumbrance of make satisfactory inferences from ensuring that the trend data from NAEP is associations found in the data between also preserved would make this task even achievement levels and school more difficult. characteristics and practices. These variables include, but are not limited to, Furthermore, while merging the two self-concept, attitudes toward school, and surveys could produce savings in total peer group attitudes and opinions. respondent burden to the educational system, it almost certainly would increase respondent burden for individual schools
58
67 and teachers that are sampled in the of a variety of student educational merged survey system. This could result outcomes and not just academic in lower response rates and threaten the achievement. To control respondent data quality for both surveys. NAEP has burden, the academic assessment tools in traditionally relied on high response rates NELS:88 had to be much shorter in scope to ensure the quality of the trend data. and content than the NAEP assessment. Again, in my opinion the integrity of these NAEP puts its burden dollars in the depth data is too important to jeopardize in a of the content while NELS:88 put its SASS/NAEP merger. burden dollars in breadth of outcomes and background information. The NAEP emphasizes the production of reliable estimates of national and state Furthermore, because NAEP does not achievement levels.Consequently, NAEP measure students longitudinally, it does not does a good, but not perfect, job of do a good job of measuring (and does not estimating the first element of student data attempt to measure) the other two elements I outlined abovemeasuring overall of my list of student data abovegrowth student achievement. However, the in achievement, and successful transitions strength of NAEP is in measuring through the increasing demands of aggregate-level measures of proficiency schooling and work. Longitudinal studies and not individual or school-level measures are needed to track these types of of proficiency. The capture of individual outcomes. For these reasons a new NELS proficiencies or achievement levels has (or some modification of NELS) may be a never been the main goal of NAEP. better candidate for merger with SASS.It Given the complex nature of the plausible is to this topic that I turn next. value methodology, individual or small group proficiencies are measured with a good deal of measurement error. Linking a New NELS with SASS
NAEP is also a survey that emphasizes While it is important to measure and track content depth over breadth of background overall achievement levels, it is also variables. The burden budget of NAEP important to be able to associate goes into accurately measuring content. differences in school policies and practices Student background coverage is not with student achievement.It is almost ignored, but certainly has less emphasis impossible to make valid inferences about than in the longitudinal studies. The the impact of school policies with cross- longitudinal studies, on the other hand, sectional dataregardless of how rich the have had somewhat different goals. For individual data may be. Of course, example, while NELS:88 also aspired to making clear inferences about these kinds provide accurate estimates of group of associations is done best by experiments proficiencies, it had the added burden of in which students are assigned to obtaining accurate estimates of school and educational treatment conditions and individual level proficiencies and subsequent growth in achievement is individual growth. There was also the measured (Metcalf, 1995). emphasis in NELS:88 on the measurement
59 68 However, true experiments in education back to the SASS schools (or are difficult to conduct and maintain under sample of SASS schools) to capture the best of circumstances. Many just those administrative records educational researchers have therefore that one needs. However,even this relied on observational survey data to would increase the administrative make cautious inferences about policy and respondent burden of the effects on achievement gains. While these survey system without providing studies have many well known inherent much in the way of analytical flaws, most educational researchers and payoff.Student test data would policymakers have been determined to not still not be available and let the "perfect be the enemy of the good" consequently measures of growth in and have conducted well thought out and achievement would also not be executed policy studies with the available. In terms of measuring longitudinal studies data systems provided transitions, one would know if by NCES (Heyns & Hilton, 1982, pp. students were still enrolled in that 89-102). school, but would know precious little else about the students' Three Options to Consider transitions to other school or work. Furthermore, some portion of the It seems to me that there are at least three students would have moved, options to attaching a longitudinal student making follow-up of their status component to SASS. These are outlined difficult and expensive. below. In addition, learning takes place in (1) Attach student administrative data to an interaction of school, home, and SASS and return to those schools to family. A student data collection pick up longitudinal data.This based solely on school records option would be substantially more obviously records only one aspect expensive than simply attaching of this learning system. The student administrative record data longitudinal studies have long to SASS since one would have to recognized this and have tried to return to the SASS schools to measure the other aspects of the follow up on the students sampled student's learning environment. in the first year. SASS is currently Measuring only one component on a five-year cycle. Presumably does not allow one to fully examine one would want to go back to the totality of the students' learning recapture student data on a more experience and how the different frequent follow-up schedule- components interact with one -perhaps every two years. Re- another. surveying schools every five years to follow up on students is perhaps (2) Create a new longitudinal survey and too long a periodicity to make "link" several items to SASS items. timely estimates of student NCES could field a new NELS outcomes. One could of course go with either an eighth-grade or
60 69 tenth-grade cohort and use identical analyses of the association of items from SASS in its school and outcomes data with school and teacher questionnaires. Linking district policy information. Data these data would provide some would also be collected with analytical payoff in terms of several follow-ups and would thus generalizabilty of the data provided. be able to measure growth in It would also decrease the burden outcomes. Information would also to individual sampled schools, be available to track the success of which would presumably not have students in making critical to respond to the both the SASS transitions through school and and NELS survey instruments. workfor example, transitions However, it would increase overall from middle school to high school, response burden and would likely through high school to high school increase overall administrative completion, and from high school costs. The analytical payoff would completion to postsecondary also be somewhat weak, since the education and/or the world of linked data to SASS would not work. include all of the contextual data provided by the new NELS. While a new NELS attached to SASS makes sense analytically, it also makes a (3) Merge a new NELS with SASS. great deal of sense in terms of cost NCES could field a new NELS in a savings. The SASS data collector will sample of SASS schools. For have already contacted the schools and example, the 1998 SASS could collected data from districts, schools, and become the base year of NELS:98. teachers. A new NELS would only have The overall analytical reward of to supplement these data with a student such a merger could be substantial. and parent questionnairethe teacher and This class of students will be on school data would be collected within the schedule to graduate in 2002, thus normal SASS administration. Using the leading to clean comparisons 1998 SASS survey as the base year of a among the high school classes of new NELS has been shown to indicate a 1972 (NLS-72), 1982 (HS&B), and substantial cost savings over a separate 1992 (NELS:88). The longitudinal sample design (J. Owings, internal memo, studies have traditionally have had 1995, National Center for Education teacher and school data, but have Statistics). not have had district-level data to attach to student data. While total response burden would Furthermore, the richness of the presumably be decreased by a NELS/SASS SASS teacher and administrator merger, the burden to individual schools data would enhance the student and will almost surely increase. However, this parent data from NELS. Student increase in response burden would have assessment data (perhaps both the potential to effect the response rates of cognitive and affective) could be the NELS data collection effort rather than attached to the SASS data to enable SASS. SASS should not have to pay any
61
70 part of the response rate price associated transition from eighth grade to high school with the merger. is a significant passage. Meaningful research has been done with the NELS:88 Thus, a new NELS attached to SASS cohort on this issue. Furthermore, data would meet the requirement that I set forth from NAEP and from NELS:88 indicate in the introduction to this paper.It would that a significant amount of cognitive and collect all three types of data that I think academic growth occurs during this are importantoverall achievement data, period. Larger gains are realized,on data on cognitive and affective growth, average, between the eighth and tenth and data on critical transitions.It would grades than between the tenth and twelfth also meet the two criteria for a reasonable grades (Crouse & Ralph, 1996). mergerit would make sense analytically, and it would make sense economically. However, despite the intuitive appeal of starting with an eighth-grade cohort, fora However, a new NELS attached to SASS variety of reasons a tenth-grade cohort would still have to overcome several may be more feasible at this time. The obstacles and several issues will need to be primary reason for this is theease with addressed in designing a new NELS. In which tenth-grade students can be followed fact, fleshing out a design for a new and therefore the lower cost involved. longitudinal study attached to SASS While younger cohorts are perhaps always deserves its own design conference. more desirable analytically than older However, short of this, I briefly outline cohorts, following younger cohorts is two areas of concern in the next section. always more expensive than following older cohorts. For example, almost 90 percent of NELS:88 eighth graders The Design of the National changed schools between the eighth and Longitudinal Study of 1998 tenth grades, while less than 20 percent of NELS:88 tenth graders changed schools What age cohort should NELS:98 begin with? between the tenth and twelfth grades. To track the transitions I outlined above, Tracking students from the eighth to the NELS:98 could start with either an eighth- tenth grade proved to be much more grade cohort (to follow the transition from expensive than originally estimated with middle school to high school and allow the NELS:88 first follow-up study. trend comparisons with NELS:88), with a tenth-grade cohort (to follow the transition Furthermore, while there was great from high school to graduation and allow analytical payoff to estimating the growth trend comparisons with HS&B and in achievement of an eighth grade in NELS:88), twelfth-grade cohort (to track NELS:88, the complexities of the the transition from high school to psychometrics involved in this effort were postsecondary education or work), or some severe. Because the NELS:88 test battery combination of the above. was used to measure overall achievement levels and growth between the eighth and Starting with another eighth-grade cohort twelfth grades, floor and ceiling effects has a lot of analytical appeal. The were much more worrisome that in HS&B,
62
71 where growth was measured between the they would want to go back and re- tenth and twelfth grades only. The interview the sampled students two years resulting adaptive nature of the NELS:88 later when most of them will be in the assessment created analytical problems twelfth grade. In this manner, trend with researchers not sophisticated with analyses could be run with the HS&B and psychometrics. For example, measuring the NELS:88 tenth- to twelfth-grade gains in mathematics proficiency was cohorts.Since SASS is currently on a much more complicated than merely five-year cycle, the two-year follow-up looking at IRT gains scores, as had been would have to be done separately from the done in HS&B. Since different kids took normal SASS cycle. These independent different tests, gains had to examined in follow-up interviews could be done either terms of gains in proficiency functioning as a CATI or as in-school interviews.In- rather than raw or IRT estimated gains. school interviews would probably be more Again, this complication was due to the costly, but would be more efficient if fact that the assessment instruments had to cognitive assessments were conducted have a multilevel design to guard against during this follow-up.(Unless someone the floor and ceiling effects that could develops a way to efficiently do a occur when testing spanned the eighth NELS:88 comparable assessment through through twelfth grades. CATI.)
It is also interesting to speculate whether a Furthermore, in many ways HS&B and twelfth-grade cohort (either selected on NELS:88 were multiple-cross-sectional their own or an "aged" tenth-grade cohort) datasets. Data were collected on the same could be attached to SASS in the high people for two years apart. What went on school years and then attached or merged in between those two data points is often in a new Beginning Postsecondary Student hard to determine. For example, detailed (BPS) survey when the year after they are information on school enrollment has been scheduled to leave high school.I realize difficult to obtain from HS&B and that the sampling issues here may be NELS:88. One knows from the various enormously complicated and can only follow-ups if sampled members were speculate about the complexities of such an attending school at the time of the follow- overlapping or multiple frame design. up, but do not know much about their However, by designing the three surveys enrollment status in between the follow-up in this manner, one would have the survey dates. One could use CATI to merged power (and savings?) of a SASS, a efficiently go back to these students more NELS, and a BPS. frequently than a two-year cycle and collect such time-sensitive data. These intermediate interviews would be limited Periodicity of SASS to just a few items (dropout status, pregnancy status, employment status) with To parallel the structure of the HS&B and fewer time dependent variables reserved NELS:88, the new longitudinal study for the more in-depth two-year follow-up should be on a two-year cycle. That is, if survey. NCES starts with a tenth-grade sample,
63 72 Summary reports engendered, a decade of educational reform took hold in the The argument for attachinga longitudinal American educational system. This component to SASS rests on several "reform" was actuallymany reforms and premises.First, attaching a longitudinal debate over the consequences of these study to SASS seems to satisfymost of the reforms continues today. NCESdata help criteria I have set out for myself.It could frame and focus this debate. measure all three of the types of student data deemed most worthwhile, while also In 1984, a cohort of students had just satisfying the two criteria for sensible graduated (in 1982) from high school. mergerproducing some cost benefit, and Their experiences in the pre-reformera engendering an analytical payoff. The would serve as a base line to judge the payoff, however, is to the overall data impact of the coming reforms. TheHigh collection effort of NCES and not School and Beyond study would recordthe necessarily to SASS data collection in experiences of this cohort of students. In particular.In fact, attaching a longitudinal 1984, another cohort of studentswas in the study to SASS may have no payoff fourth grade. These students would feel whatsoever for SASS but may indeed some of the immediate consequences of provide more burden to the already these reforms. Their experiences in high overworked SASS staff.Attaching school, in postsecondary education, and in aggregate longitudinal student data to the transition to the world of workwere SASS may be of more benefit to SASS captured in the experiences of the students itselfmerging a new NELS and SASS in the National Educational Longitudinal provides the most benefit to NCES and Study of 1988. In 1984 (theyear in which indeed, to the whole educational policy A Nation At Risk made its first impact),yet community. another cohort of childrenwere born who are right now experiencing the full impact of the reforms of the last two decades. Conclusion Most of this cohort areon track to graduate from high school in 2002. The years 1983-84 saw the release oftwo publications that would forever change the Unfortunately, current budgetconcerns way that Americans looked at their cast doubt on whether NCES will be able elementary and secondary schools. Ernest to field an independent longitudinal study Boyer's High School: A Reporton of this class of high schoOl students. The Secondary Education in America, 1983 cohort of students who will be included in focused public attention on American high the Early Childhood Longitudinal Study schools, a "troubled institution" witha will not be graduating from high school confused mission and low standards. At until 2012. Missing the class of 2002 will about the same time the U.S. Department result in a data gap of almost 20years and of Education released A Nation at Risk, will weaken our ability tomeasure the which called attention to whatwas termed impact of the changes introduced intoour a "rising tide of mediocrity" in American elementary and secondary schools.Failing schools. Due in part to the publicity these to capture the experiences of the high
64
73 school class born at the very beginning of Crouse, J., & Ralph, J.(1996). The reform will be a serious gap in the nation's National Educational Longitudinal Study: knowledge about education. Linking a Race and gender differences in academic new longitudinal study with SASS may be achievement. Paper delivered at the the only way of effectively filling this data American Educational Research gap. Association meeting, New York.
Heyns, B., & Hilton, T. L.(1982). The References cognitive tests for high school and beyond: An assessment. Sociology of Education, Arnold, C., & Kaufman, P.(1992, June). 55, 89-102. School effects on educational achievement in mathematics and science: Metcalf, C. E.(1995). Incorporating 1985-86 (NCES 92-066). Washington, experimental designs into new NCES data DC: U.S. Department of Education, collection methodologies. In G. National Center for Education Statistics. Hoachlander, (Ed.), From data to information: New directions for the Burstein, L., & Aschbacher, P.(1987, National Center for Education Statistics December). Further thinking on the (NCES 96-901). Washington, DC: U.S. merger of the National Assessment of Department of Education, National Center Educational Progress and the Schools and for Education Statistics. Staffing Surveys: Summary and recommendations for two meetings of Pallas, A. (1993, Winter). Schooling in statisticians and researchers. Unpublished the course of human lives: The social manuscript, Center for Research and context of education and the transition to Evaluation, Standards, and Student Testing adulthood in industrial society. Review of at University of California, Los Angeles. Educational Research, 20.
1. This quote is attributed to Dan Koretz (and similar sentiments attributed to Richard Murnane and Marshall Smith) in L. Burstein and P. Aschbacher (1987).
2. A weak measure of SES has been used in several analyses using the NAEP data. See C. Arnold and P. Kaufman for an example.
3. This happened in the mid-1980s with the NAEP reading assessment, resulting in the so-called "reading anomaly."
65 74 MAKING DATA LEVANT FOR POLICY DISCUSSIONS: RECOMMENDATIONS FOR REDESIGNING THE SCHOOL ADMINISTRATO QUESTIONNAIRE FOR THE 1998-99 SASS'
'miry Y. Zheng, Ohio State University
Introduction significant implications for their tracking and on-the-job performance (p. 166)." As an integral part of the Schools and Moorman and associates' observation Staffing Survey (SASS) conducted by the touches upon a sensitive issue that has long National Center for Education Statistics been debated within the education (NCES), the School Administrator administration community. However, Questionnaire has been used to collect without the support of authoritative information from both public and private evidence, such an observation remains school administrators regarding their subjective and inconclusive.Fortunately, demographic characteristics, academic with the implementation of SASS, policy background, professional training, and issues such as this can now be fully attitudes toward school management issues explored by tapping into the data resources (Davis & Sonnenberg, 1995). Thus far, collected through the school administrator three surveys were conducted separately in survey. Within NCES, both Hammer and 1987-1988, 1990-1991, and 1993-1994. Rohr's report (1994) on the disproportional These efforts have resulted in a large distribution of male and female principals integrative database that can be used to in America's public and private schools present a comprehensive national profile of and Rossi and Daugherty's report (1995) school administrators as a professional on the types and locations of schools at work-force. Research products based on which America's minority principals work this data source have already provided have rendered strong statistical evidence to valuable information to education decision- support some of the arguments made by makers on a number of important policy Moorman and associates (1992). issues. The school administrator survey, together For example, in their report to the with other components of SASS, not only National Leadership Network, Moorman provides data for mapping the basic and associates (1992) argue that there is a demographic and educational background pervasive bias favoring white male of school administrators, it also offers principals over female and minority opportunities for assessing principals' principals in America's schools. They attitudes toward school management issues question whether female or minority such as the priorities of educational goals, principals may "inhabit a school different seriousness of school problems, and the from their majority counterparts" and distribution of decision-making power in whether this difference may "hold schools. As education reform continues to demand improvements and greater
67
75 accountability from our educationalsystem An Overview of the School to better prepare students for future challenges, it becomes evident that Administrator Questionnaire of SASS information regarding school principals will continue to be of great interestto education researchers and policymakers. The school administratorsurvey of SASS As Odden (1995) points out, the is the most comprehensive and thelargest decentralization of decision-makingpower national study of school principals in this from school boards to individual schools country, perhaps even in the world. The has placed school principals at the only study outside of NCES thatcan forefront of the current reform movement. competently approach the depth and It is therefore important thatwe have extensiveness of SASS is thesurvey of reliable and comprehensive informationon school administrators by Feistritzer and the principalship in order to make sound associates in 1987 for the National Center judgments on school administrators' roles for Education Information.Feistritzer and and contributions to school improvement associates' study (1988) surveyed the basic and student outcomes. demographic background of school administrators and their attitudes towarda NCES is currently in the process of number of school management issues. The study has a sample of more than five reviewing the direction, purposes, anduses of SASS for the planned 1998-99survey. thousand elementary and secondary school This includes examining the currentuses administrators, including superintendents, of its data, its relationships with other public and private school principals and federally sponsored data collection efforts, assistant principals. Compared to the and future national survey needs in school administrator survey of SASS, accordance to changing policy priorities. Feistritzer and associates' study hasa number of disadvantages. As part of the review efforts, thispaper It is a one-shot will discuss the scope, uses, and possible study, hence it is limited by its inabilityto changes of the school administrator provide a longitudinal perspectiveon component of SASS from a user's changes occurring within the school perspective.In the following chapters, I administrator workforce. It also lacks the will first present an overview of the school broad scope that SASS has. Feistritzer and administrator questionnaire across three associates' study only provide basic separate surveys. Next, an assessment of demographic information suchas gender, the current uses of the school administrator age, education, and work experience.It does not have other equally important data will be given.Lastly, comments and suggestions for possible changes to the information such as principals' school administrator questionnaire will be professional training and the contextual offered. environment of schools in which principals conduct their daily business. Moreover,
68
76 Feistritzer and associates' survey did not 1995). Secondly, the school administrator offer user-friendly data resources to other survey is structurally integrated with other researchers for further exploring the policy components of SASS. For every school issues related to the principalship. included in the survey, its principal and a number of teachers within the same school Compared to Feistritzer and associates' would also be surveyed. The school's file and other similar studies, data collected is also linked with the school district's file. from the school administrator surveys of These inter-file linkages provide a high SASS have several distinct advantages. degree of flexibility to data users for First of all, it has a large and incorporating relevant variables from other comprehensive sample of principals from databases. For example, while the school all varieties of schools.It includes not survey provides contextual information only principals from public schools of regarding the 'schools in which principals different sizes, locations, and levels, but fulfill their leadership roles, the teacher also private schools of different group survey supplements additional information types and religious affiliations. The 1987- on how well principals perform such 88 SASS has a sample size of 9,317 public leadership roles (from teachers' schools and 3,513 private schools (NCES, perspective). Moreover, the school 1994) while the 1990-91 SASS has a administrator questionnaire has maintained sample size of 9,330 public schools and a high level of consistency over the past 3,270 private schools (Kaufman & Huang, three surveys that many of the core items 1993). Such a high degree of remain unaltered. Such a consistency representation affords researchers the allows researchers to evaluate the changes opportunity to conduct analysis down to over time in many areas of the the basic level of the stratification principalship. Policymakers may use such sampling structure. For example, there data to assess the changes and progresses are even enough cases for comparing three of the school administration workforce. different types of Jewish schools in the private school sample (McLaughlin et al.,
69 77 Table One An Overview of QuestionnaireItems of School AdministratorComponent of The Schools and Staffing Surveysat NCES
Number of Items' Categories of Questions Specific Types of Questions Asked 1987-88 1990-91 1993-94
Education and Training Degrees achieved and major fields of 25 18 292 study
Professional Training 4 7 6 Professional Experience Teaching experience: years and 6 4 4 assignment fields
Administrative experience: years 5 5 243 and positions
Other job experience 6 6 17° Career plan Plan to remain as principal 2 6 Compensation Salary 2 2 2
Benefits 10 10 10
Demographics Gender, age, race 4 5 5
Job-related Activities Activities and hours spent 11
Perceptions Perceptions of school problems 13 22 24
Perceptions of influence on school 9 15 395 matters
Perceptions on school educational 3 3 goals
Questions about school's Teacher evaluation 1 3 -
Teaching staff Teacher training 1 -
Teacher recruitment 8 -
Miscellaneous Data & telephone number 3 3 4 Total 108 105 173
1 Refers to total number of response items. A question may have multipleresponse items. 2 BA/BS degree granting university and location were added. 3 Grade level of previous principal positions and breaks in principalcareer were added. 4 New position categories and years of experience were added. 5Private school version has 27 items.
70 In Table One, an overview of all survey extensive study of the gender factor on items is presented for the school principals' career decisions, their job administrator questionnaires. While the performance, their compensations, and total number of survey items for the first their job satisfaction, Gross and Trask two surveys is about the same (108 and (1975) documented significant differences 105 respectively), the number of items for between male and female principals the 1993-94 survey increases substantially through personal interviews and mail to 173. Given the busy schedule of school surveys. Twenty years after Gross and administrators, it is reasonable to believe Trask's study, one wonders how our that this total number of question items has education system has been changed to reached the critical length. Any increase achieve gender equity. To answer this or in length will very likely cause a decline in other related policy questions, these core survey return rate. Across the three data items from the school administrator questionnaires, there are eight general surveys are particularly useful. categories of questions: questions about the education and training of principals, their Principals' perceptions of school matters professional experience, their plan to retire are another general category of question or to remain as principals, their salary and items. The school administrator benefits, their perceptions on a variety of questionnaires ask principals to indicate school matters, their basic demographic their perceptions regarding the seriousness background, their job-related activities, of a variety of problems facing their and their opinions of their teaching staff. schools, their perceptions over the distribution of decision-making influence at Among these eight general categories of the school, and their ranking of important questions, items inquiring about principals' educational goals.' Over the years, these education and training, professional items regarding principals' perceptions experience, compensation, and basic have proliferated. The items for mapping demographic background remain consistent principals' perceptions on school problems throughout all three surveys. These increased from 13 items in 1987-88 to 24 questions are the core items of the items in 1993-94. Items regarding questionnaires. They are essential in principals' perceptions on the distribution tabulating the gender, age, and racial of decision influence for school matters distributions of school principals and in (also called "locus of control" items) presenting a basic profile of their increased from nine items in 1987-88 to 39 educational background and professional items in 1993-94.3 To a certain extent, preparation (including pre-service, in- such increases reflect efforts committed by service training and work experience). the staff at NCES to make the SASS The availability of these data items enables surveys more relevant to the policy debates the tracking of the dynamic changes in the over reform and restructuring in education basic characteristics of our nation's administration.If we are to put more education administration workforce.It power into the hands of school principals also helps answer some of the fundamental and teachers to decide what is good for questions about the reform of the principal their schools and their students, we ought workforce itself.For example, in an to know what they think about the merit of
71
0 0 the reform and the impacts of its School Administrator Research implementation. Understandably,most of Using SASS An Assessment these perceptual items could also be found in the teachers' questionnaires. The three waves of SASS school administrator surveys have accumulated Question items about principals'career a large amount of data about school plans, such as decisions to retire or to principals in the United States. Modern remain as principals, grow fromnone in 1987-88 to six items in 1993-94. These decision-support theory believes that data can be transformed into information only items help shed light on the supply and when it is used to assist decision-making demand situation of the school (Rohrbaugh, 1986; Hammond & Arkes, administrator workforce. Together with 1986). According to this view, the information about principals' ages and career breaks, these data items can be used effectiveness of data-collection efforts is to assist the projection of demands for ultimately judged by the quality of the data new and how the data are used to inform policy principals.Judging by the fact that decisions. In a report prepared by the information regarding principals' supply and demand situation is seriously lacking, Research Triangle Institute, Curtin and Fiore (1995) clearly indicate that the the availability of these items can be quite valuable to those who have stakes in school administrator database from SASS is a very useful source of information for training and recruiting new school education decision-makers. administrators. In a sequence of topics, Curtin and Fiore demonstrate how the school administrator data While most categories of questions in the can be used to inform policymakers and education school administrator questionnaires have researchers about the pluralistic experienced increases in question items transformation of the principalship in over time, two categories of questions America's schools, the changing have been down-sized. Questions about qualifications and professional preparation principals' job-related activities and of school principals, the questions about the recruitment and new roles required for principals in managing evaluation of teachers were actually schools, and the status of principal eliminated from the 1993-94 survey. retention and turnover. These changes may have been justifiedat the time the 1993-94 surveywas planned. However, it is my opinion that these items should be restored but in different formats. Details will be discussed in Section IV of this paper.
72 SO Table Two Samples of School Administratoresearch Using NCES Data
Research Based on SASS Research Based on HS&B
Type of Author, Year, Title Publication Author, Year, Title Type of Publication Hammer, C. and Rohr, C. (1993) Issue brief Chubb, J. and Moe, T. Conference paper Teaching, Administrative, and Other NCES Publication (1985) Politics, Market, and American Political Work Experience of Public School the Organization of Schools Science Association Principals Hammer, C. and Rohr, C. (1994) Issue brief Eberts, R. and Stone, J. Journal article Public and Private School Principals? NCES Publication (1988) Student Achievement Economics of Are There Too Few Women? in Public Schools: Do Education Review Principals Make a Difference?
Rossi, R. and Daugherty, S. (1995) At Issue brief Hannaway, J. and Talbert, J. Research Paper Which Types of Schools Do Minority NCES Publication (1991) Bringing Context into OERI Publication Principals Work? Effective Schools Research: Urban-Suburban Differences
Ingersoll, R. and Rossi, R. (1995) Who Issue brief Goldring, E. and Rallis, S. Book Influence Decision-making About NCES Publication (1993) Principals of Corwin Press School Curriculum: What Do Principals Dynamic Schools: Taking Say? Charge of Change Anderson, J. (1993) Who's Runs the Research Report Lee, V. et al. (1993) Journal article Schools? The Principal's View OERI Publication Teachers and Principals: Educational Gender-related Perceptions Evaluation and of Leadership and Power in Policy Analysis Secondary Schools
Fowler, W. (1991) What Are the Conference paper Brewer, D. (1993) Journal article Characteristics of Principals Identified AERA Principals and Student Economics of as Effective by Teachers? Outcomes: Evidence from Education Review U.S. High Schools
Haller, E. et al. (1994) Does Graduate Conference paper Education in Educational AERA Administration Improve America's Schools? Another Look at Some National Data Ballou, D. and Podgursky, M. (1995) Journal article What Makes A Good Principal? How Economics of Teachers Assess the Performance of Education Review Principals
73 81 ITEM' COPY AVAU_ABLE In Table Two, research products using the principal-related research products based SASS principal database are listed together on SASS data was accepted for external with research products using the High publication. The relativesuccess of HSB- School and Beyond (HSB) principal based research products is partly dueto database. HSB is a national longitudinal the early inception of the HSB database. survey of high school sophomores and The fact that HSB datawere collected seniors conducted also by NCES. Students almost four years earlier than SASS data selected to participate in the studywere gave HSB data-users much more time to administered a series of cognitive tests get familiarized and to work with the data. measuring their verbal and quantitative Another reason that may explain the skills in 1980. Several follow-upsurveys relative success of HSB-based studies is were conducted with sub-samples of the that principal and teacher questionnaires original sample population to determine were added to the original HSB survey of changes in their test scores.In 1984, an students' cognitive abilities witha clear "Administrator and Teacher Survey" intention of linking principals' (ATS) was added to the HSB study, with performance to student outcomes. This questionnaires administered to principals, addition enabled the merging of the guidance counselors, and teachers in about demographic background and personal 500 schools, or about half of the original perceptions of principals and teachers with number of sample schools (Chubb & Moe, students' test scores. This merging 1985). The added ATS was intended to provides the convenience and opportunities study the organizational aspects of schools for exploring the connections between --schools' relationships with parents and principal-related variables and student school boards, teachers' perceptions of outcomes. principals' leadership roles, and principals' perceptions of school environments and In a study of principals' influenceon management practices. Many of the student outcomes based on HSB data, question items used in ATS were later Brewer (1993) used the change in student incorporated into the teacher and principal test scores between sophomore year (1980) components of SASS.It is therefore and senior year (1982) as a dependent useful that in discussing the use of SASS variable to measure the influence of data to study the principalship thatwe also principals on student outcomes. His study discuss research products that are basedon reveals that principals can influence HSB data. student outcomes through the setting of academic goals for students, through the Overall, it appears that research endeavors screening of new teachers, and through based on HSB data had enjoyed greater their decisions on instructional methods. success in getting their products accepted This study, together with the study by by external publications (see Table Two). Eberts and Stone (1988), is one of thevery Research works using HSB data were small number of empirical studies on accepted not only by academic journals principals' influences on student outcomes and conferences, but also by a major book that were reportedly backed by direct publisher that specializes in education- statistical evidence. Brewer claims that his related topics.In contrast, only one of the findings render supports to the "effective
74
82 schools" argument that principals can 1984 as in 1980 or 1982?Moreover, make a difference in student's outcomes there were drop-outs in the test population (Brewer, 1993). between 1980 and 1982.It may be reasonable to speculate that these drop-outs Despite the optimism, research findings are probably among the students who did based on the linking of ATS data with not perform well in the first cognitive test. HSB data are not without their perils. First Did this selection bias have influence on of all, the connection between principal the internal validity of the research behaviors and student performance is findings? indirect at best (Glasman & Heck, 1992; Kleine-Kracht, 1993).Principals do not In comparing the ATS of HSB and the interact with students directly and their principal and teacher components of SASS, influences on student achievements are Ingersoll (1995) points out quite clearly muted by other more powerful factors, the differences between the two databases. such as the quality of teachers, the degree ATS was developed specifically to of parental involvement, and students' own facilitate the investigation of relationships motivations, just to name a few.It is between school staffing characteristics and difficult to imagine that these variables can student outcomes (Chubb & Moe, 1985) be controlled in the analysis. Therefore, while SASS is intended to provide a caution needs to be exercised in making comprehensive assessment of schools and direct inference from principals' staffing conditions in the United States. performance to student outcomes. ATS has a usable sample of about 350 Furthermore, the use of students' cognitive secondary schools while SASS has more test scores as the sole basis for assessing than 12,000 sample schools of different student outcome is too simplistic.Student levels. ATS has a limited generalizability outcome is a multi-dimensional construct. of schools due to its small sample size To reduce such a broad concept into a while SASS supports national estimates by single dimension solely based on test any number of different school scores undermines the moral and social characteristics, including sector, level, values of education, not to mention that state, urbanicity, and school size. SASS is test-based criteria can also be quite also more accurate in distinguishing complicated. The way Brewer (1993) used different types of private schools. For the data also led to some unsettling example, SASS separates private schools questions about the reliability of his into sub-types grouped by their religious findings. For example, the dependent orientations and group affiliations. variable he used is the change in student test scores between 1980 and 1982, but the The comprehensiveness of the SASS data independent variables are from principal and the availability of three waves of and teacher surveys conducted in 1984.In surveys have provided ample opportunities light of the time differences, we need to for conducting in-depth analysis on a ask: Are those principals surveyed in 1984 number of key education policy issues. the same principals in those schools With direct relevance to education between 1980 to 1982? Did teachers and administration research, we may use the principals have the same perceptions in data to study the changing demographic
75 83 characteristics of the education effectiveness.His findings indicate that administration workforce and how these principal effectiveness is a complex and demographic changes are associated with multi-dimensional construct anda changes in salary and compensations. principal's ability in leading the school Policy studies such as these can answer effectively is influenced bya number of questions on whether economic equity for factors, including the principal'sage, women and minority principals has been gender, teaching experience or the school's improved as their shares in the location, size, and level.Similarly, the administration workforce expanded. We study conducted by Haller, Brent, may also pursue studies to understand the McNamara, and Rufus (1994) also used perceptual differences between principals teachers' perceptions of principals to and teachers regarding the decision-making create indices of school leadership, but structures in schools and how such Haller and associates' interest is to find differences in perceptions are conditioned out how graduate training in educational by their educational and demographic administration would help improve background. Such studies may help principals' leadership effectiveness. The explain the trends and patterns of findings from Haller and associates' study decentralization and the locus of control in lead to a disturbing yet tantalizing school management. Or, we may want to conclusion: graduate training in education find out whether the effectiveness of administration does not have significant principal leadership is constrained by the influence on the attributes that characterize organizational settings or the socio- effective principals. Judging by the fact economic conditions of schools.The that most states require a master's degree study of principal leadership using SASS or even a doctoral degree in Education data, even without the direct linkage to Administration as a prerequisite for student outcomes, can still shed light on principal licensure, this finding posts how schools can be more effectively serious questions on the validity of sucha governed and improved. requirement to education policymakers. Recently, the Los Angeles Unified School Over the past several years, there has been District decided to waive this and other a number of principal-related research mandated requirements for hiring new papers that based their findings on the principals,' hence setting an example for analysis of the SASS data.In addition to challenging the status quo of school the issue briefs and research reports administration licensing practices. This published by the National Center for bold reform effort is clearly supported by Education Statistics, there are two what Haller and associates discovered from academic conference papers and one analyzing the SASS data. published journal article that employed the SASS data to address issues surrounding In the only externally published research the principalship and school paper that employed SASS data to address administration. For example, Fowler principalship issues, Ballou and Podgursky (1991) used teachers' perceptions of (1995) used the 1987-88 SASS data to principals' leadership behaviors to create evaluate the influence of principals' an index of perceived principal educational credentials and professional
76
84 experience on teachers' assessment of into a Windows-based system to enhance principals' leadership practices.Ballou its user-friendliness.' This change is and Podgursky find little statistical imperative now given the fact that new evidence to support the recent proposals Windows operating systems such as for enhancing the "professionalism" of the Windows 95 or Windows NT are phasing principal workforce by requiring more out DOS-based software.Alternatively, advanced degrees and additional SASS can create its own world-wide web administrative training. They argue that (WWW) home-page to provide online, 24- principal licensing requirements in the hour access to the database.' states may undervalue those attributes that characterize good school leaders.For Efforts to promote the awareness and example, principals with more teaching access of the SASS database will provide experience are generally rated higher by the necessary conditions for wider use of teachers. However, most states only the data resources for education research. require new principals to have a few years However, in order to facilitate the use of of teaching experience. Ballou and the data to conduct policy analysis, Podgursky's study obviously raise serious modifications must be made to the content questions for policymakers to ponder. of the questionnaire so that it is more relevant to current policy debates.In the Despite the relatively small number of following section, I will discuss my studies using SASS to examine the roles personal view on how to improve the and practices of school principals, those principal questionnaire. that had been completed have provided many interesting and fresh insights on policy issues related to school Improving the School administration. Given the great potential Administrator Questionnaire: of SASS as a comprehensive national Some Suggestions database on schools and their staffing patterns, it is imperative that more studies In a discussion of methods for assessing be done to take advantage of the depth and the effectiveness of public funded research richness of the database.In order to organizations, Altschuld and Zheng (1995) encourage more researchers to use the believe that a stakeholder-based approach SASS data to study education policy is more useful than a goal-based approach issues, NCES must play a more active role in assessing the performance of research in facilitating the awareness of and access organizations. This is because research to the dataset.For example, a brochure organizations usually have broad and describing the database together with general goals and their organizational suggested research questions can be outputs are mostly intellectual products distributed to directors of graduate studies and services that cannot be measured at universities to encourage doctoral meaningfully in tangible monetary or students to utilize the database for writing numeric terms. Performance of research their dissertations. The electronic organizations thus are better gauged from codebook system now available with the their customers' perspective.In the case SASS CD-ROMs should be transformed of SASS, its customers would include
77 85 education researchers, planners and actually echo with some of the policymakers at federal, state, and local community's prevalent views. Murphy governments, and individuals and and Beck (1994) believe that principals' organizations who have interest in schools roles and responsibilities must be clarified and school staffing issues. at a time when schools are forced to transform and restructure.Despite the In order to improve the relevance and increased importance of their jobs, usefulness of the data products of SASS, it principals themselves have been silent and is logical that we find out what the passive in defining their roles in school customers' current and future needsare. leadership. The clear definition of To this end, I decided to obtainsome first- principals' roles will help principals hand knowledge by conducting a small understand how to meet the demands for survey via the Internet discussion group accountability, how to adapt to the "AERA-A7" hosted by Arizona State changing social fabric of schools, and how University.' In my e-mailsurvey, I to make schools meet the needs of a post- asked, "what is the most important policy industrial world.In a widely cited paper, research issue for education administration Re-Thinking School Leadership: An Agenda in the next 10-15 years?" Of the 18 for Research and Reform, Bolman, answers with direct relevance to the Johnson, Murphy, and Weiss (1990) of the principalship, 28 percent of the answers Harvard Graduate School of Education (n=5) indicated that principals' roles and make similar observations. They believe contributions in school decentralization and that school leaders, under the constraints restructuring should be the most important of changing student demographics, fiscal research issue; 33 percent of the answers difficulties, and greater societal (n=6) stated that principals' accountability expectations, must learn to cultivate to school outcomes should be the key constituent supports, including support issue.Specifically, one respondent wanted from parents, teachers, and school boards. to know how principals can be evaluated School leaders should adapt tonew fairly through demonstrated added value to management reality and to establish the learning process of students; 22 mutually dependent and cooperative percent of the answers (n=4) identified the relationships with teachers. School leaders working conditions and career decisions of should also understand how the technology principals as the major research concern. of instruction can facilitate the delivery of Lastly, 16 percent of the responses (n=3) knowledge from teachers to students. rated principals' attitudes and handling of new information technologies in classroom Most of the recent studies on principals' teaching and learning as the most roles in school restructuring and reform important issue. are built on the framework of the "effective school" movement (Murphy & This survey may be too small to collect the Louis, 1994; Hallingert & Leithwood, opinions of the broad research community 1994).Despite the marginal variations in of education administration; however,a their views, these studies in generalagree review of the literature shows that the with Murphy's (1994) argument that research issues raised by these respondents school restructuring produceda nearly
78
86 overwhelming workload for principals, Core Items demanded that they work both harder and smarter, and created considerable work- The core items of the questionnaire, that related stress.In order to survive these is, questions about principals' education new challenges, principals must adapt to and professional training, work roles changes in several important areas: experience, compensations, and delegating more decision-making power to demographic background should be teachers and other support staff and retained for all surveys. These items are promoting a collaborative relationship with deemed essential, for they provide the them; enabling and supporting teacher necessary conditions for constructing the success through more constructive basic profile of the school administrator approaches in professional developments; workforce. Policy issues may change and managing a constellation of change efforts, research emphases may shift over time, including the direct involvement in but the need for understanding the basic instructional practice improvement; and characteristics and working conditions of extending the school boundary through school principals is continuous. active community marketing efforts to Researchers, policymakers, and other cultivate parental and public support concerned parties need these core data (Bookbinder, 1991; Elmore, 1995; Odden, items to track the dynamic changes in our 1995). nation's education administration workforce.Policy research based on the Judging from the above discussions on the analysis of perceptions, school outcomes, important policy research issues for or community opinions also need to education administration in the years to consider the contextual constraints of the come, it is quite clear that principals' new principalship.Therefore, it is imperative roles in leading schools at a time of that these items be retained for all surveys. change and uncertainty is of major concern to education researchers and policymakers. Nevertheless, some of the questions in the If SASS is to contribute more to these core items are too elaborate and some of policy discussions, its survey them are too simple. Changes should be questionnaires must be updated from its made to balance the two extremes. current format to become more relevant to Specifically, questions about principals' the needs of the educational policy education background may be too research and decision-making elaborate. For example, it may not be communities.In light of these discussions necessary to ask whether a principal has an and the objectives of SASS, I would like associate's degree and in what field. Given to make the following suggestions for the fact that the majority of principals has bringing the current SASS school at least a master's degree and nearly administrator questionnaire (1993-94 everyone has a bachelor's degree, what is version) to tie more closely to the current the value of knowing whether one has an policy debates: associate's degree?'It is also not necessary to ask about the location and name of the college from which they earned their college degrees. There is no
79 87 written rule that a principal must graduate certification and preparationprocess, it is from an Ivy League school or a top-tier therefore necessary thatwe ask more public university. Knowing the schools questions about them.In a recent article from which they graduated will not on principal training programs, Bjork and contribute significantly to the Ginsberg (1995) criticize the conventional understanding of the policy issues approach to principal training as too surrounding the principalship. abstract from reality to be of real-lifeuse in practice. They believe that principal Question #16, "were you the principal of training programs in the United Statesare this school in the Spring of 1991?" is in need of a paradigm realignment, that is, redundant since the question right after it fundamental changes that will force the asks the respondents, "prior to this school entire field to shift from academic-oriented year, how many years were you employed to practice-oriented training. These as the principal in this school?"If the changes may include sending university answer is greater than two, by logic, the faculty members to schools to gain clinical respondent would most probably be the experience or release principals from principal of this school in the Spring of school-level duties to pursue full-time 1991.Question #18b asks, "in what training that will integrate their school grade levels were the students in the administrative experience with theories. school in which you last served as principal?" There are 15 choices, ranging Observations made by researchers (Bjork from pre-kindergarten to grade 12.Each & Ginsberg, 1995; Ballou & Podgursky, choice takes up one data space.In order 1995; Haller et al., 1994) regarding to increase the efficiency of the answer principal training programs clearly indicate format, it would be better that a smaller that much needs to be learned about the range of choices was used in this case. content and impact of these programs. In For example, instead of listing all possible order to facilitate the policy changes for grade levels, the question can simply have improving these training programs, the several general categories of answers (i.e., principal survey of SASS should expand elementary, secondary, elementary and the number of questions in this area.In secondary combined, and others). addition to asking whether a respondent had participated in a training program,we In addition to formal education, principals should also ask how long the training usually receive other types of training to lasted, how frequently he/she received the prepare themselves for the job as a school training, and how satisfied was he/she with leader.Exactly what kinds of training did the outcome of the training.For they receive and how effective was that example, we can ask: training in helping them prepare for the challenges ahead? We have no way to Have you participated in any in-service know, for the questions on professional training in evaluation and supervision? training are simply too vague (only "yes" Yes No or "no" answers are available). Since pre- service and in-service training are very important parts of the principal
80
88 If yes, how many times have you had this significantly high percentage of missing type of training during your entire answers (one possibility is that principals principal career? did not keep good track of exactly how Times (give a number) they spent their time and were unwilling to venture a guess).If these problems are To what extent did this training help you real, they would definitely create problems become a more effective principal? to the reliability of these data items, hence Extremely helpful Very helpful their elimination is the sensible thing to Somewhat helpful Not helpful do. Waste of time However, the elimination of these items Another area that should be modified is the created a regrettable void in the principal questions about service breaks in a questionnaire of those items about the jobs principal's career.Instead of asking and responsibilities that principals whether the breaks were due to layoff or a performed. It is regrettable because reduction-in-force, multiple choices should modern principalship is such a complex be offered. For example, reasons for enterprise that most outsiders really cannot breaks in services could be: layoff, comprehend the extent to which these jobs organizational restructuring, educational and responsibilities burden school leave, personal/sick leave, military leave, principals.' If we don't ask these and others. Through multiple choices, we questions in our surveys, we miss the best may be able to know more about the opportunity to understand how school reasons why principals have to leave their principals perceive and perform their jobs. Additionally, we should ask how responsibilities. The need to know more long it took them to find a new principal about the jobs principals perform is greater position if they were unable to return to than ever. As my previous discussions on their original positions. This information the key policy research issues indicate, would shed light on the demand and principals presently are given more supply situation for school principals. responsibilities as education reform, political changes and technological Principals' Jobs and Responsibilities improvements have shifted more decision- making power from districts to schools. In the 1987-88 SASS survey, principals With the increase in responsibilities, calls were asked to give their best estimates of are heard with increasing frequency for the number of hours they spent on several greater accountability and more extensive categories of school-related activities, i.e., performance review for principals (Kirst, teacher supervision or curriculum 1990). management. In the two SASS surveys that followed, this entire set of questions Paradoxically, the increase of was eliminated. Through my responsibilities does not come with better conversations with the staff at NCES, I got understanding of principals' jobs and their the impression that it was eliminated due abilities to perform those jobs. to questions about the reliability of Gottfredson and Hybl (1987) provide a principals' self-reported numbers and a very good observation on this paradoxical
81 89 situation. They believe that much of the roles and responsibilities are obviously the demands for principals' increasing type of data that have a very high degree accountability to school effectiveness are of policy relevance to decision-making. A "based on very limited knowledge of what major study of school administrators such principals actually do and which aspects of as SASS simply cannot ignore this critical the job are most important and most aspect of the principalship. burdensome. Furthermore, although much writing and advice on the principalship is In order to include question itemson generic, the role of the principal may principals' jobs and responsibilities in the differ according to the kind of school the questionnaire and not to repeat the pitfalls principal leads. Most principals must of the 1987-88 SASS survey, the questions learn the ropes on the job with limited must be framed differently.Instead of support and guidance. Many schools do asking principals to provide estimateson not have a clear written job description to time usage, we may ask them to rank the spell out what is expected of the principal importance of a number of jobs related to (p. 1)."Clearly, to know more about the their management responsibilities and how jobs and responsibilities of principals they actually allocate time to accomplish should be an important goal of a national those jobs. For example, we can ask survey of school principals.If data are principals questions in the following collected for making informed policy format: decisions, then, data about principals'
Among the following school-related activities, please provide us your ratings of their importance to your jobas a school leader and the time you spent on them given your current workload:
Activities Ira orptTIce Time Spent Sit in a classroom to observe 0-Not a part of my job 0-None teachers' instruction 1-Not important 1-Little 2-Somewhat important 2-Occasionally 3-Moderately important 3-Frequently 4-Very important 4-Extensively (a major part of my job)
Talk to parents about their 0-Not a part of my job 0-None children's school problems 1-Not important 1-Little 2-Somewhat important 2-Occasionally 3-Moderately important 3-Frequently 4-Very important 4-Extensively (a major part of my job)
Take actions to ensure enough 0-Not a part of my job 0-None computer & telecommunication 1-Not important 1-Little equipment for students" 2-Somewhat important 2-Occasionally 3-Moderately important 3-Frequently 4-Very important 4-Extensively (a major part of my job)
By presenting questions in this format, we jobs and responsibilities. The scales for can avoid asking principals to pinpoint the the answers can be fine-tuned to better exact number of hours they spent on each capture the importance and the time spent activity and to preserve the opportunity to on each activity.Further studies are also obtain valuable information about their needed to find out what activities should
82
90 be included in the list.But it is certain such influence to a great extent have to that these activities should represent those rely on teachers' performance as a medium jobs and roles typically performed by to deliver the effects. principals in their capacities as the cultural, managerial, instructional, moral, Although the teacher component of SASS and strategic leaders (Leithwood & Duke, has already provided large amount of data 1993). Not all jobs and responsibilities on teachers' quality, they are from are viewed as equally important to teachers' own perspective. The addition principals. Some responsibilities may be of a few items in the principals' viewed as less important but would questionnaire will give us an additional consume more of their time. And some perspective on teachers' quality.Since we responsibilities may be viewed as have asked teachers to evaluate the important but they are unable to devote performance of their principals, we should more time to do. If we need to know how also ask principals to tell us how they feel principals can effectively improve their about their teachers. Current efforts to schools, we at least should know reform our schools call for principals to something about how they perceive their work more closely with their teaching staff roles and responsibilities and how much to improve student outcomes.In order to time they have to spend on each activity. assess how the collaborative relationship between principals and teachers can Principals' Perceptions of Their Teaching flourish and how such a collaboration Staff affect the overall effectiveness of the school, it is useful that we gain an In the 1987-88 SASS survey, principals understanding of both principals' and were asked about the availability of formal teachers' perceptions of the other party. evaluation systems for teachers.In the 1990-91 SASS survey, this set of questions Principals' Perceptions on School Matters was changed to solicit principals' perceptions on the quality of their teaching In all three waves of SASS surveys, staff.However, in the 1993-94 SASS, principals and teachers were asked to these questions were removed completely. reveal their perceptions on the seriousness Given the usefulness of these items and the of a range of school problems, issues fact that there are only 2-3 items for this related to decision influence (locus of set of questions, its removal is also quite control) on school matters, and the regrettable.Slater and Teddlie (1992) importance of a number of educational believes that an effective school must goals.In the 1993-94 survey, the total possess three key components: teacher number of items for the perception of preparedness, student readiness, and school problems is 24 for both principal administrative appropriateness. These and teacher questionnaires.In addition, three components must be integrated into the public school principal questionnaire an unbroken chain of actions in order to has 39 "locus of control" items while the generate better school outcomes. private school principal questionnaire has Principals may have influence on 27.Since SASS already has three surveys, improving students' achievements, but in retrospect, judging by the frequency of
83 91 the items being used and the consideration restructuring the organizational for reducing some questions in order to arrangements for school governance. make room for new items, I would like to Since teachers and principals are suggest that the entire section on increasingly charged with more authority principals' perceptions of school problems in determining the curriculum, personnel, be removed from the next survey and the and discipline policies of schools, it is items regarding "locus of control" be naturally necessary that both parties' retained. perspectives be considered.
I believe that the central objective for Issues regarding the organizational knowing principals' and teachers' arrangements for power sharing in schools perceptions on school problems such as are sensitive yet important. Despite the student tardiness and student drug use is to obvious reasons for principals to work provide policymakers and researchers data closely with teachers to achieve school on how school administrators and teachers outcomes, there are many problems that feel about the problems facing schools. may lead this collaborative relationship to Information about these perceptions can falter. Wooster (1991) believed that part alert the public and decision-makers to of the problem could be attributed to each give higher priorities to support principals party's perception of their domain of and teachers to solve these problems. influence. For example, teachers may feel Since teachers interact directly with that they should have the most say in students and have first-hand knowledge of instructional matters. Therefore, when a students' conditions inside and outside of principal visits a teacher's classroom to the classroom, we would assume that they observe instructional practice and make at least have equally valuable comments on comments on possible improvements, the school problems. Teachers' perceptions teacher may have the impression that the may be different from principals'. But for principal is interfering with his/her right to the purpose of understanding school teach and is imposing an administrator's problems facing students inside and outside view on the teacher who may be a better of classrooms, teachers' perceptions should expert on the subject.Other issues that be sufficient to help inform us of the can be explored with these "locus of seriousness of those problems. control" items are the differences between private and public school principals and The "locus of control" items are a between private and public school different matter, for the central objective teachers. In a survey of Catholic teachers, here is to find out the perceptions on the Kushner and Helbling (1995) point out that distribution of decision influence among a private school teachers tend to agree more number of people. Knowing the with their principals on school differences between principals' and management issues and that such teachers' perceptions help researchers and agreements are mostly based on mutual policymakers understand the decision- trust, while such trust and agreement are making and organizational structures in much weaker among public school teachers schools. Such an understanding in turn toward their principals. How true is this can help evaluate current efforts in observation? Does this difference
84 92 contribute to the cultural differences devote some attention to this issue.In the between public and private schools? We questionnaire, we may ask principals when can find out some answers by comparing they obtained their first principal license the perceptions between public and private and at what level, how many renewals did school principals and teachers. they have after the first license, whether they needed to apply for a new license Principal Preparation and Licensure when they transferred from an out-of-state administrative position, or whether the As I mentioned earlier in this paper, licensing process helped them become information regarding principals' pre- more effective school leaders. service and in-service training and preparation are not detailed enough to provide good estimates on the impacts of Conclusion these training and preparation programs on principals' leadership effectiveness.In In conclusion, I would like to reiterate the addition to my previous suggestions for importance and usefulness of the principal expanding these questions, I would also surveys of SASS in contributing to the like to see the inclusion of several understanding of the characteristics and questions regarding principals' licensure in conditions of the school administrator the next SASS survey. Almost all states workforce in America. As the most require principals to possess a legitimate comprehensive study of school principals school administration license and to renew currently available, the principal survey of the license after a period of time in SASS has provided valuable data for service. Reading through the job exploring various important policy issues advertisements for principals, one cannot regarding the basic characteristics of help but notice that a principal's license is school principals in the United States, always one of the most important including their education background, prerequisites for the job. Given such an professional training, work experience, emphasis on principal licensure, one has to salary and compensation, and their wonder whether such a requirement has perceptions on a range of school been helpful in keeping the principal management matters. workforce to a higher standard; or did the licensing process keep some of the In order to further extend the principal brightest minds from the teacher workforce survey's utility in educational policy or other professions away from this debates, I have suggested above a number important and challenging field of of changes to the principal survey leadership? questionnaire. These suggestions include: keep the core items consistent throughout Despite the relevance of principal licensure all survey efforts but simplify those items practice to the formation of the principal that are overly elaborate; expand the items workforce, information regarding this on principals' in-service and pre-service practice is scarcely available.It is training programs and solicit principals' therefore useful that in at least one of the level of satisfaction with those programs; principal surveys of SASS that we can request that principals rank the relative
85 importance of a number of school Anderson, J.(1993). Who runs the activities as related to their role as school schools? The principal's view (Research leaders and ask how they allocate their Report No. OR-93-3078). Washington, time for those activities; remove DC: U.S. Department of Education, principals' perceptions of school problems Office of Research. to make room for new items; retain the "locus of control" items; and include some Ashbaugh, C. R., & Kasten, K. L. questions in the next survey regarding (1992). The licensure of school principal licensure procedures. administrators:Policy and practice. Washington, DC: American Association These suggestions are based on my of Colleges for Teacher Education. understanding of the major policy research issues for education administration in the Ballou, D., & Podgursky, M. (1995). near future. The changes I suggested do What makes a good principal? How not include possible items to evaluate how teachers assess the performance of principals can create "added value" principals? Economics of Education directly to student achievement as Review, 14(3), 243-252. suggested by some scholars in my e-mail survey.It is not the objective of the Bjork, L. G., & Ginsberg, R.(1995). Schools and Staffing Survey to assess the Principals of reform and reforming immediate impact of principals on student principal training: A theoretical outcomes.It is also my contention that perspective. Education Administration principals' influence on students' learning Quarterly, 31(1), 11-37. are indirect as long as teachers are the ones who teach in the classroom. Given Bobbitt, S., & McMillen, M. (1995). these constraints, it is natural that Qualifications of the public school teacher principals' demographic characteristics, workforce:1988 and 1991 (NCES 94- their educational and professional 665). Washington, DC: U.S. Department backgrounds, their perceptions of school of Education, National Center for management issues, their perceptions of Education Statistics. their teaching staff, and their economic status should be the major concerns of a Bolman, L. G., Johnson, S. M., Murphy, national survey of school principals. J. T., & Weiss, C. H. (1990). Re- thinking school leadership: An agenda for research and reform (Occasional Paper References No. 1).Cambridge, MA: The National Center for Education Leadership: Harvard Altschuld, J. W., & Zheng, H.(1995). Graduate School of Education. Assessing the effectiveness of research organizations: An examination of multiple Bookbinder, R. M. (1991). The approaches. Evaluation Review, 19(2), principal: Leadership for the effective and 197-216. productive school.Springfield, IL: Charles C. Thomas Publisher.
86 94 Brewer, D. J.(1993).Principals and Fiore, T. A., & Curtin, T. R.(1996). students' outcomes: Evidence from U.S. Public and private school principals in the high schools.Economics of Education United States: A statistical profile, 1987- Review, 12(4), 281-292. 88 to 1993-94. Washington, DC: U.S. Department of Education, National Center Choy, S., Henke, R., Medrich, E., & for Education Statistics. Bobbitt, S.(1993). Schools and Staffing in the United States: A statistical profile, Fowler, W. J.(1991). What are the 1990-91 (NCES 93-146). Washington, characteristics of principals identified as DC: U.S. Department of Education, effective by teachers? Paper presented at National Center for Education Statistics. the meeting of the American Educational Research Association, Chicago, IL. Chubb, J. E., & Moe, T. M. (1985). Politics, markets, and the organization of Gilat, A., & Sulzer-Azaroff, B.(1994). schools. Paper presented at the annual Promoting principals' managerial meeting of the American Political Science involvement in instructional improvement. Association, New Orleans, LA. Journal of Applied Behavior Analysis, 27(1), 115-129. Davis, C., & Sonnenberg, B.(Eds.). (1995). Programs and plans of the Ginsberg, R., & Thompson, T.(1993). National Center for Education Statistics Dilemmas and solutions regarding (NCES 95-133). Washington, DC: U.S. principal evaluation. Peabody Journal of Department of Education, National Center Education, 58-74. for Education Statistics.
Eberts, R. W., & Stone, J. A. (1988). Glasman, N. S., & Heck, R.(1992). Students achievement in public schools: The changing leadership role of principals: Do principals make a difference? Implications for principal assessment. Economics of Education Review, 7(3), Peabody Journal of Education, 68(1), 5- 291-299. 24.
Elmore, R. F.(1995). Teaching, Goldring, E. B., & Rallis, S. F.(1993). learning, and school organization: Principals of dynamic schools. Newbury Principles of practice and the regularities Park, CA: Corwin Press. of schooling.Education Administration Quarterly, 31(3), 355-374. Gottfredson, G. D., & Hybl, L. G. (1987). An analytical description of the Feistritzer, C. E., Quelle, F., & Bloom, I. school principal's job.Baltimore, MD: (1988).Profile of school administrators in The Johns Hopkins University Press. the United States. Washington, DC: National Center for Education Information. Greenfield, W. D. (1995). Toward a theory of school administration: The centrality of leadership.Education Administration Quarterly, 31(1), 61-85.
87 95 Gross, N., & Trask, A. (1975). Men and Ingersoll, R. (1994). Organizational women as elementary school principals. control in secondary schools. Harvard Cambridge, MA: Harvard Graduate Educational Review, 64(2), 150-171. School of Education. Ingersoll, R.(1995). An agenda for Haller, E. J., Brent, B. 0., McNamara, research on teachers and schools: J., & Rufus, C.(1994). Does graduate Revisiting NCES's Schools and Staffing training in education administration Survey (NCES 95-18). Washington, DC: improve America's schools? Paper U.S. Department of Education, National delivered at the meeting of the American Center for Education Statistics. Educational Research Association, New Orleans, LA. Ingersoll, R., & Rossi, R. (1995). Who influences decision-making about school Hal linger, P., & Leithwood, K. (1994). curriculum: What do principals say? Exploring the impact of principal (Issue Brief, NCES 95-780). Washington, leadership. School Effectiveness and DC: U.S. Department of Education, School Improvement, 5(3), 206-218. National Center for Education Statistics.
Hammer, C. H., & Rohr, C. L. (1993). Kaufman, S., & Huang, H. (1993). Teaching and administrative work Schools and Staffing Survey, 1990-91: experience of public school principals Sample design and estimation: Technical (Issue Brief, NCES 93-452). Washington, report (NCES 93-449). Washington, DC: DC: U.S. Department of Education, U.S. Department of Education, National National Center for Education Statistics. Center for Education Statistics.
Hammer, C. H., & Rohr, C. L.(1994). Kirst, M. W. (1990). Accountability: Public and private school principals: Are Implications for state and local there too few women? (Issue Brief, NCES policymakers. Washington, DC: U.S. 94-192). Washington, DC: U.S. Department of Education, Office of Department of Education, National Center Educational Research and Improvement. for Education Statistics. Kleine-Kracht, P. A. (1993). The Hammond, K., & Arkes, H. (1986). principal in a learning community. Judgment and decision making: An Journal of School Leadership, 3(4), 391- interdisciplinary reader.Cambridge, 399. U.K.: Cambridge University Press. Kushner, R., & Helbling, M. (1995). Heck, R. H., & Marcoulides, G. A. The people who work there: The report of (1993, May). Principal leadership the Catholic Elementary School Teacher behaviors and school achievement. Survey. Washington, DC: National NASSP Bulletin, 77, 20-28. Catholic Education Association.
88
96 Lee, V. E., Smith, J. B., & Cioci, M. Murphy, J., & Beck, L. G.(1994). (1993). Teachers and principals: Gender- Ethics in educational leadership programs: related perceptions of leadership and An expanding role.Thousand Oaks, CA: power in secondary schools.Educational Corwin Press. Evaluation and Policy Analysis, 15(2), 153-180. Murphy, J., & Louis, K. S. (Eds.). (1994).Reshaping the principalship: Leithwood, K., & Duke, D. L.(1993). Insights from transformational reform Defining effective leadership for efforts. Thousand Oaks, CA: Corwin Connecticut's Future Schools. Journal of Press. Personnel Evaluation in Education, 6(4), 301-333. National Association of Elementary School Principals.(1990).Principals for 21st Leithwood, K., & Steinbach, R.(1993). century schools. Alexandria, VA: Total quality leadership: Expert thinking National Association of Elementary School plus transformational practice.Journal of Principals. Personnel Evaluation in Education, 7, 311- 337. National Center for Education Statistics. (1991). SASS and TFS Questionnaires Lipham, J. M. (1981).Effective 1987-1988.Washington, DC: U.S. principal, effective school.Reston, VA: Department of Education, Office of National Association of Secondary School Educational Research and Improvement. Principals. National Center for Education Statistics. Marcoulides, G. A., & Heck, R. H. (1992). SASS and TFS Questionnaires (1993). Examing administrative leadership 1990-91. Washington, DC: U.S. behavior: A comparison of principals and Department of Education, Office of assistant principals.Journal of Personnel Educational Research and Improvement. Evaluation in Education, 7, 81-94. National Center for Education Statistics. McLaughlin, D., O'Donnell, C., & Ries, (1994a). An overview of the SASS and L.(1995).Private schools in the United TFS (NCES 94-440). Washington, DC: States: A statistical profile, 1990-91 U.S. Department of Education, Office of (NCES 95-330). Washington, DC: U.S. Educational Research and Improvement. Department of Education, National Center for Education Statistics. National Center for Education Statistics. (1994b).Quality profile for SASS: Moorman, H. N. (1992). Strengthening Aspects of the quality of data in the support and recruitment of women and Schools and Staffing Surveys (SASS) minorities to positions in education (NCES 94-340). Washington, DC: U.S. administration. Washington, DC: U.S. Department of Education, Office of Department of Education, Office of Educational Research and Improvement. Educational Research and Improvement.
89
97 National Center for Education Statistics. Silins, H. C. (1994). The relationship (1994c). SASS and PSS Ouestionnaires between transformational and transactional 1993-1994 (NCES 94-674). Washington, leadership and school improvement DC: U.S. Department of Education, outcomes. School Effectiveness and Office of Educational Research and School Improvement, 5(3), 272-298. Improvement. Slater, R., & Teddlie, C. (1992). Odden, A. R.(1995). Educational Toward a theory of school effectiveness leadership for America's schools.New and leadership. School Effectiveness and York: McGraw-Hill, Inc. School Improvement, 3(4), 242-257.
Ogawa, R. T., & Bossert, S. T.(1995). Stufflebeam, D., & Nevo, D. (1993). Leadership as an organizational quality. Principal evaluation: New directions for Educational Administration Quarterly, improvement. Peabody Journal of 31(2), 224-243. Education, 24-46.
Pajak, E., & McAfee, L.(1992). The The National Commission on Excellence in principal as school leader, curriculum Education.(1983). A nation at risk: The leader. NASSP Bulletin, 76(Nov.), 21-30. imperative for educational reform. Washington, DC: U.S. Department of Pantili, L., Williams, J., & Fortune, J. Education, the National Commissionon (1991). Principal assessment: Effective Excellence in Education. or not? Paper presented at annual meeting of AERA, Chicago, IL. Thomson, S. D.(Ed.).(1992). School leadership: A blueprint for change. Rohrbaugh, J.(1986).Institutional Newbury Park, CA:Corwin Press. research as decision support.In J. Rohrbaugh & A. T. McCartt (Eds.), Valentine, J. W., & Bowman, M. L. Applying Decision Support Systems in (1991, Dec.). Effective principal, Higher Education. San Francisco: Jossey- effective school: Does research support Bass Publishers. the assumption. NASSP Bulletin, 75, 1-7.
Rossi, R., & Daugherty, S.(1995). van de Grift, W., & Houtveen, T.(1991). Where do minority principals work? (Issue Principals and school improvement. Brief, NCES 96-840). Washington, DC: School Effectiveness and School U.S. Department of Education, National Improvement, 2(1), 53-70. Center for Education Statistics. Wooster, M. M. (1991).First principals: Rowan, B., Raudenbush, S. W., & Kang, The leadership vacuum in American S. J.(1991).Organizational design in schools. Policy Review, 57(Summer), 55- high schools: A multilevel analysis. 61. American Journal of Education, 68, 238- 266.
90
98 1. I would like to express heartfelt appreciation to the following people who made suggestions for improving this paper:Kerry Gruber, Charles Hammer, Sharon Bobbitt, and Daniel Kasprzyk of NCES, Ramsay Selden of Education Statistical Services Institute, and John Mullen of Policy Studies Associates.
2. The 1987-88 survey did not include the item for ranking educational goals.
3. Thirty-nine items for public school questionnaire; 27 items for private school questionnaire.
4. "Matter of Principal," Editorial, Los Angeles Times, page 8, November 2, 1995.
5. In my opinion, the DOS-based electronic codebook system is somewhat difficult to install and this initial problem may discourage many new users from exploring the data CDs.
6. NCES is currently in the process of putting the SASS data on the World Wide Web siteat http://www.ed.gov/NCES. The Electronic Codebook for SASS 1987-88 is currently available.
7. AERA-A is a group organized by the Education Administration section of the American Educational Research Association and has members from the research, practice, and policymaking communities of education administration.
8. Records of the survey are archived at Internet site "magnus.acs.ohio-state.edu".Electronic copies are available upon request by sending an e-mail to yzheng @magnus.acs.ohio state.edu.
9. In the 1993-94 SASS, only 4.9 percent of principals indicated that they had an associate's degree.
10. One of the reasons that the principal questionnaire is much shorter than the teacher questionnaire is the consideration that principals are under greater time and work pressures.
11. This question partially addresses one of the policy issues raised by several respondents in my e-mail survey about principals' support for new technologies in schools.
91 99 MEASURES OF INSERVICE PROFESSIONAL DEVELOPMENT: SUGGESTED ITEMS FOR THE 1998-1999 SCHOOLS AND STAFFING SURVEY
Dorothy M. Gilford
PART I (7) How much effort are teachers expending on IPD in terms of time Inservice Professional Development and their own money? in the United States (8) How much support/encouragement are they receiving for IPD in the What information do we need about form of incentives, financial inservice professional development? support, and time for IPD? Without attempting to be comprehensive, a number of questions immediately come to (9) How prevalent are induction mind: programs for beginning teachers, what areas are addressed in these How is inservice professional (1) programs, who provides support, development (IPD) planned and how effective are the programs? coordinated? (10) What is the level of public sector (2) Is the school environment investment for IPD and what is it supportive of IPD? purchasing? What is the range of programmatic (3) (11) Are there better ways to invest approaches? these resources? (4) What are teachers doing to (12) How are the characteristics of IPD strengthen their practice--what is changing over time? the format, location, length, and content of their IPD programs?1 (13) What can we learn from IPD in other countries (especially those What are the teachers' perceptions (5) whose students do well in of the effectiveness of the IPD international assessments) that programs in which they participate? might help improve IPD in the United States? (6) How effective are the IPD programs in improving teaching (14) What changes are needed in and in enhancing students' inservice professional development learning? to meet the challenges of the current systemic reform movement in the United States?
93
10 0 Information relevant to most of these consideration of a large potential addition questions can be obtained from the 1998- to SASS on this topic in 1998-1999. 1999 SASS. Exceptions are questions 6 and 10.Smaller, sharply focused studies Part I starts by considering various would be more appropriate for studying definitions of IPD and its evolution--which the effectiveness of IPD programs in is reflected in the definitions and themany improving teaching and in enhancing types of IPD programs they encompass. students' learning. Nor is it feasible to Turning to the design of SASS items, it is estimate the level of public sector noted that the large number of types of investment for IPD from a survey of IPD programs calls for a framework to schools and staff, since information would organize information collection and be needed from many sources other than compilation. Such a framework is the schools, e.g., federal, state, and proposed and is used in the development district agencies. The subject of question of items related to the prevalence of IPD 10, induction programs, is not addressed types.Since data from the 1998-1999 in this paper, but will be the subject ofa SASS will become available in theyear later paper. 2000, the target year for measuring the effects of reform in meeting the goals of To respond to the other questions, itseems the Goals 2000 program, the national and appropriate at this early stage of the state reform initiatives and their development of the 1998-1999 SASS to implications for IPD are discussed. A set suggest some possible items for SASS that of reform-oriented approaches for IPD is would provide information related to the then presented, as are some characteristics questions. The items will not be defined of effective programs. Part I then in great detail, since the purpose is to considers the principles of high-quality stimulate discussion about the value and IPD programs and uses these principles to feasibility of including the items in SASS- develop a number of items related to an approach that is consistent with the quality. The final section of Part I purpose of this paper which is to addresses several data needs for the Year recommend items about IPD for possible 2000 National Education Goals Report. inclusion in the 1998-1999 SASS. Throughout the paper, when an item is suggested that would provide information What Is IPD? related to the questions, a footnote indicates the related questions. We first consider several different definitions of IPD followed by a brief The first part of the paper is limited to description of the evolution of IPD, IPD in the United States, while Part II concluding with the type of IPD needed discusses the value of international for successful reform. comparisons of IPD generally and in particular for use of computers and Definitions of IPD advanced telecommunications equipment. The focus on IPD for computers in this The Department of Education defines paper is to provide ample time for professional development as including "the
94
1 0 1 rigorous and relevant strategies and directions of the district and the school, organizational supports that insure the and is targeted to specific issues rather career-long development of teachers and than across an array of disconnected other educators." areas." This mode of staff development is not only useful in reform efforts, but, as It includes preservice preparation and will be seen later in the paper, it is training of teachers as well as inservice consistent with the current consensus of professional development. This paper is IPD experts about the principles of good limited to inservice professional IPD. development, i.e., to activities designed to maintain or upgrade teachers' professional Bellanca (1995) distinguishes among skills following certification or inception inservice, staff development, and of teaching including the induction period. professional development from the systems point of view: The Education Information Network in the European Union and the EFTA/EEA Inservice is the scheduling of Countries (EURYDICE) defines inservice awareness programs, usually of training as "...a variety of activities and short duration, to inform teachers practices in which teachers become about a new idea in the field of involved in order to broaden their education. knowledge, improve their skills and assess and develop their professional approach" Staff development is the effort to (Perron, 1991). correct teaching deficiencies by providing opportunities to learn A somewhat different definition of IPD new methods of classroom was proposed by Orlich:"Programs or management and instruction. activities that are based on identified needs; that are collaboratively planned and Professional development is a designed for a specific group of planned, comprehensive, and individuals; that have a very specific set of systemic program designed by the learning objectives and activities; and that system to improve all school are designed to extend, add, or improve personnel's ability to design, immediate job-oriented skills, implement, and assess productive competencies, or knowledge" (Orlich, change in each individual and in the 1989, p. 5) school organization.
A different view of staff development is From the individual's point of view, provided by Odden and Marsh (1988, p. Bellanca notes that"...professional 598), who are concerned with reform of development begins with the individual's secondary schools:"The emerging mode election to expand his or her repertoire of of staff development addresses broader and knowledge or skills" in a program "that more complex issues, is provided over helps the individual understand and do longer time periods with considerable higher quality teaching." ongoing assistance, is linked to strategic
95 102 Evolution of IPD constructivism (Sparks, 1995). According to Sparks these ideas are causing changes The definitions of IPD reflect its in IPD. Today IPD is moving towards evolution. As described by Bellanca, individual development and organizational many years ago inservice opportunities development; it is driven bya strategic were limited primarily to annual institutes plan for the school district, each school, at which teachers reviewed basic topics for and the departments that serve schools; it annual relicensing. At a later date schools is school focused rather than district and districts introduced the workshops and focused; focuses on student needs and conferences that are now so prevalent. learning outcomes; involves multiple forms of job-embedded learning; focuseson a Staff development programs differ from combination of generic and content- these inservice events in that these new specific skills; is a major responsibility programs required 20 to 30 hours' study of performed by all administrators and the theory and description of the practice teacher leaders; is concerned with (e.g., some of the science and mathematics continuous improvement in performance programs that were introduced after for everyone who affects student learning; Sputnik). Many staff development and is an indispensable process for programs in the 1980s and the early 1990s preparing students for citizenship and dealt with cooperative learning approaches productive employment. or with thinking-skills. Although some districts are moving in In the early 1990s staff developers began these directions, most districts are to investigate ways to match professional continuing past practice.In the schools development with school improvement; to today we can find all three types of IPD move away from teaching methods that defined by Bellanca (inservice, staff might improve learning and to move development, and professional toward management systems that would development) including programs thatare ensure raised test scores. They recognized mixtures of the types. Therefore SASS that the constructivists' insights apply to questionnaires need to cover all of them. professional development as well as to To avoid confusion, this paperuses the students' learning.District leaders began term IPD as comprising the three types. to understand the power of systemic Although there is no consensus about the support systems that communicate the idea best type of professional development, the that learning as a lifelong process isas view of staff development described by important for the teachers as it is for the Odden and March, Bellanca's description students. of professional development, and the changed IPD described by Sparks Today the schools and IPD are being correspond to the type of IPD experts shaped by three ideas:results-driven consider to be essential for successful education, systems thinking, and reform.
96
103 A Framework for Classifying participation in any of eight activities Types of IPD Programs related to teaching and the other concerning participation in programs that As the number of approaches to IPD focused on each of five topics and the proliferates, it becomes increasingly duration (in hours) of the program: important to have a systematic way of classifying the approaches in order to 30.Participation in types of collect and collate information about IPD inservice activities.Since the end systematically. A framework for of last school year, in which of classifying IPD types will be useful in these activities related to teaching developing survey items about teachers' have you participated? (1) staff development activities and in SCHOOL DISTRICT sponsored analyzing the resulting data. The workshops or inservice programs, framework should be sufficiently general (2) SCHOOL sponsored to cover the IPD activities of teachers workshops or inservice programs, during their induction period as well as (3) University extension or adult those of experienced teachers, although education courses, (4) College some of the specific types of activities courses in your subject field, (5) within the framework categories might Professional growth activities differ for the two groups of teachers. For sponsored by professional example, during the induction period associations, (6) Committee to teachers might have a mentor, a program integrate academic skills into of visiting and observing experienced vocational education, (7) Other teachers, a lighter work load, or regular curriculum committee, (8) meetings with senior staff and other Committee on selecting textbooks beginning teachers. Experienced teachers or materials, (9) None of the might take college courses to update their above. knowledge of their subject matter field or Participation in programs with recent research on pedagogy. 31. a specific focus.Since the end of Before proposing the framework, three last school year, have you different approaches will be considered: participated in any inservice or building on the categories used in the professional development programs 1993-94 SASS, using the five models of that focused on the following staff development proposed by Sparks and topics?(a) Uses of educational Loucks-Horsley (1990); or using the six technology for instruction (e.g., use research-based models proposed by Gall of computer, satellite learning), (b) Methods of teaching your subject and Vojtek (1994). field, (c) In-depth study in your IPD Categories Used in the 1993-94 SASS subject field, (d) Student assessment (e.g., methods of The SASS Teacher Questionnaire included testing, evaluation, performance the following two questions on types of assessment), (e) Cooperative staff development: one concerning learning in the classroom. For
97 104 each yes answer there is a question what his/her learning needsare and "How many hours did the program establishes a goal. The teacher last?" with three options:8 hours chooses from workshops, library or less, 9-32 hours, or more than research visits, courses offered by 32 hours. the district, or may be reimbursed for college courses he/she takes, Sparks and Loucks-Horslev Models and other forms of self study to reach the goal. Sparks and Loucks-Horsley (1990) proposed five models of staff development. Observation/assessment: these forms The five models were based on their of IPD include clinical supervision, analysis of strategies that share similar peer coaching and teacher assumptions about "where knowledge evaluation with particular attention about teaching practice comes from" and to certain behaviors and open "how teachers acquire or extend their discussion of the results. knowledge". Loucks-Horsley and her colleagues (1987) assert that staff School development/improvement development programs that are effective in processes: (This term is used by changing teachers' behavior have common Darling-Hammond and Cobb; characteristics. They combine theory and Loucks-Horsley describes this as application, they provide time for curriculum and program reflection and practice and involve self development.) states, districts, or study and cooperative learning. The five schools that try to improve models are described by Loucks-Horsley education implementing whole- and her colleagues (1989) and Darling- school change recognize the Hammond and Cobb (1995): importance of teachers as agents of change. Teachers participate in Training: e.g., workshops school improvement activities, sponsored by schools or districts curriculum and assessment where an expert makes a development, and shared decision- presentation focusing on knowledge making structures.(Little [1993] and skills teachers are lacking. commented that teachers often learn This is the most common model.It more through school development includes (1) development of the processes than through more theory and rationale behind the new traditional staff development behaviors to be learned, (2) activities.) demonstration or modeling, (3) practice in the training setting, and Inquiry: includes such activities as (4) guided practice in the field with teacher study groups, teacher feedback on performance (Loucks- research, teacher Horsley et al., 1989). collaboratives/networks, or reflective inquiry. Such activities Individually guided professional stem from the reform efforts that development:the teacher judges view the teacher as a guide or
98 105 facilitator of students' active ability to reflect and make sound learning, which forces teachers to decisions formulate questions about teaching and learning and to inquire both Skill-training:trainer presents into students' thinking and learning theory underlying the skills, and the effects of their teaching. explains, and models the skills. Teacher practices skills and Loucks-Horsley and her colleagues (1989) receives feedback, is coached to provide a detailed description of examples promote transfer of training to own of the actual implementation of each of classroom. (Consistent with the these five types of staff development. constructivist movement in education that assumes that Gall and Vojtek Models individuals learn best when they are given responsibility for developing Gall and Vojtek base their six models on their own knowledge and the objectives of professional development understanding.) Objectives: described by Sparks and Loucks-Horsley. development of teachers' (1) These models are described in terms of the instructional skills and strategies; different roles for staff developers, and in (2) ability to improve students' ascending order of their complexity: academic achievement; (3) ability to develop and implement Expert-presenter: teachers assemble curriculum; (4) ability to reflect to listen to an expert talk about a and make sound decisions topic at professional conferences, school district assemblies, Action-research: teachers do university courses, and how-to research in their own work setting workshops. Although this is the to answer their questions or test a most prevalent model, it is not new idea.Objectives: changing powerful in itself; it needs to be teachers' attitudes; development of used in conjunction with other teachers' ability to engage in school models. Objectives: development restructuring of teachers' knowledge and understanding Organization-development:a coherent, systematically planned, Clinical-supervision: the change- sustained effort at system self-study process supervisor, mentor, or and improvement focusing coach identifies a teacher's explicitly on change in formal and concerns and goals, collects informal procedures, processes, classroom observation data, and norms, or structures, and using reviews data with the teacher. concepts of behavioral science. Objectives: development of The goals of organizational teachers' instructional skills and development are to improve strategies; development of teachers' organizational functioning and performance. Therefore it focuses
99 106 on groups of teachers and other Objective: development of school staff. An organization- teachers' ability to engage in school development specialist helps restructuring teachers and other staff diagnose strengths and weaknesses of their Proposed Framework school or system, develop a plan of action, implement the plan, and The types of inservice activities in the evaluate its success.Objectives: SASS questions are muchmore specific changing teachers' attitudes; than the categories of types in thetwo sets development of teachers' ability to of models and can easily be fitted into develop and implement curriculum either typology. Since it is desirableto use general categories in a framework, Change-process: the goal is a only the Loucks-Horsley and Gall-Vojtek systemic innovation requiring models were considered in the proposed change at the school or district framework, which consists of seven level.Staff developers help models. The models and the source of teachers make a decision to adopt a each model follow: systemwide innovation, put the innovation into action, and Expert-presenter: (Gall and Vojtek) institutionalize it.Initiation This model was and may still be requires staff development to get the most common form of IPD.It teachers to buy into change; staff has been severely criticized by IPD development required for experts as relatively useless for implementation includes "concrete, reform. Nonetheless it will be teacher specific training activities, important to ascertain the extent to ongoing continuous assistance and which it persists in 1998-1999. support during the process of This model was not proposed by implementation, and regular Loucks-Horsley. meetings with peers and others." Institutionalization, the decision to Skill-training: (both typologies) continue using the systemic innovation indefinitely, requires Observation/assessment: (both staff development to ensure that the typologies) innovation continues to be used as intended--helpful to have teachers Individually guided professional and other educators who are highly development: (Sparks and Loucks- skilled in the innovation and who Horsley). Gall and Vojtek did not can provide training and support to include this model since they new staff.According to Fullan described their models in terms of (1991), this is by far the most the roles for staff developers. complex and lengthy of the models, requiring three-to-five years for Inquiry: (Sparks and Loucks- moderately complex changes and Horsley) This model includes five-to-ten for major changes. action-research, which was a
100 107 separate model in the Gall and comment. This draft describes the Vojtek typology, and encompasses research base for linking student learning, many more types of activities, e.g., teaching quality, and professional teacher collaboratives/networks,2 development; discusses professional and reflective inquiry. development indicators; proposes a typology for the indicators; and reviews Organization-development: (Gall and some 25 surveys for questions that Vojtek) This model and the correspond to the indicators. When following "change-process" model agreement on the typology has been are combined by Sparks and reached, they plan several additions to the Loucks-Horsley. They are clearly draft:a display of the questions from the separable, "organization- 25 surveys; identification of the elements development" corresponding to of professional development that are efforts to improve the performance important, measurable, and representative; of teachers within an existing and a prioritization of these elements. system, and "change-process" to changing the performance of Although the framework of models teachers in a systemic innovation at proposed in this paper can easily be fitted the school or district level. into Mullens' typology, it has not been done because of the typology's tentative Change-process: (Gall and Vojtek) state.Instead, suggestions are made for Because of the current emphasis on SASS to use the proposed framework of systemic reform, it is desirable to models to collect data on participation in be able to measure the prevalence professional development by type and of "change-process" professional related items about the types. Two items development. are suggested to replace items in the 1993- 94 SASS.
Application of the Framework to Prevalence of IPD by Type, Time Teachers Develop SASS Items on Prevalence Spend in Each Type, and Total Dollars Teachers Spend on IPD of IPD Question 30 in the 1993-1994 SASS can be Before suggesting specific items, the expanded to provide time teachers spend recent work of others related to SASS on each type of IPD. items should be recognized. Mullens (1995) reviewed measurement approaches Item 1:3 Prevalence of IPD by Type, for classroom instructional processes.In Time Teachers Spend in Each Type, and 1996, Mullens and his colleagues Total Dollars Teachers Spend on IPD. undertook a comprehensive look at the The stem might be worded "Since the end theoretical linkages and current of the last school year, how many hours measurement of student learning, teaching have you spent in each of the following quality, and professional development. types of staff development? The item They have released a preliminary draft of should list the various types of their work (Mullens et al., 1996) for
101
108 professional development activities under Another proposal made by Mandel (1995) each of the seven major models. By is related to Item 1. He proposed that providing columns corresponding to time NCES undertake a set ofcase studies on a intervals in SASS question 31 and regular basis that would provide portraits including a column for zero time, dataon of the range of programmatic approaches prevalence of participation in types of IPD being undertaken in continuing education as well as the time spent in the programs (also in preservice education.) Henotes can be obtained. At the end of the item, the messiness of measuring post-licensing add the question "How much of yourown education since it takes place in teacher money have you spent on IPD during this centers, colleges and universities, school school year?" (This last question was districts, seminars run by disciplinary and added as a reaction to Mandel's (1995) specialty groups, and in other informal statement that "...the extent to which settings such as seminars. Nonetheless he teachers meet their employers halfway is considers this an arena that is crucialto no less important" than the way schools the health of the profession,one that invest their resources for IPD.) deserves much more attention than it has received. Question 30 in the 1993-1994 SASS provides a list of eight types of Although such case studies could not easily professional development programs. be a part of the SASS surveys, they could These should be included as subcategories well be part of the development work that of the framework in the question for the would help define SASS questions about next SASS to provide trend data. the range and character of IPD.
The list should include other types that Program Content and Length, and have been prevalent in the past (e.g., Teachers' Perceptions of Program Impact committees dealing with subjects other than curriculum, workshops sponsored by It is possible to build on Question 31 in the school system during the summer, the 1993-1994 SASS to obtain information skill-training workshops, conference on the content of IPD programs. The attendance, made a presentation at a SASS question obtained informationon conference or other professional meeting, the duration of programs focusedon five participation in special projects, scheduled topics. Three of them were topics related consultation with colleagues, and to current types of methodological independent reading). The reform-oriented instruction important in reform: uses of approaches discussed in a subsequent educational technology for instruction, section should also be included. Mullens' student assessment, and cooperative ongoing review of IPD items in over 25 learning in the classroom. The other two educational surveys may also produce were types of knowledge identified by additional types. The ultimate list will be Shulman (1986) as necessary for expert long, but the question should not be too teaching--content knowledge and burdensome to teachers. pedagogical content knowledge. (Shulman also named pedagogical knowledgeas a third type of essential knowledge.). An
102 10,9 important topic missing from this set is portions of Question 32 on impact of the "classroom management skills," described programs as described above. by Mullens et al. (1996) as explaining rules, monitoring behavior, using Teachers are not alone in judging the accountability systems to keep track of quality of IPD programs. A number of students' work, communicating experts and several organizations have expectations clearly, and maximizing the provided sets of principles of effective IPD amount of class time available for programs. academic work.It would also be useful for the Year 2000 National Education Goals Report to add the topic "teaching Education Reform and Teacher limited English proficient (LEP) students" Inservice Professional Development to the set covered in this question. A fuller discussion of this topic is included in National Reform Initiatives a later section on the goals report.It would also be useful to add topics for Education reform has been pervasive in the teaching other types of special student United States since 1983 when the first populations such as multicultural classes or wave of reform was generated by the classes that integrate special education publication of A Nation at Risk (National students. Commission on Excellence in Education, 1983). Since preservice and inservice It is possible to combine Question 32 with professional development are important Question 31 and obtain teachers' opinions elements of education reform, it is about IPD programs in each of the seven important to measure the characteristics topics proposed for Question 31. The and prevalence of professional yes/no participation question in Question development as fully as possible to 31 can be eliminated by adding a "0 understand the extent of these elements of hours" category on the right side of the reform. Several reform activities are question. This leaves space on the left for discussed to illustrate the pervasiveness of the stub of Question 32 and for three education reform in the United States. columns: agree, no opinion, and disagree. Although this sacrifices the more detailed In 1986, a second wave of reform scale in Question 32, it has the advantage followed the 1983 wave. This second of removing the ambiguity in Question 32 wave was stimulated by reports from a that was created by not being able to number of organizations including the differentiate among IPD programs. California Commission on the Teaching Profession, the National Governor's Item 2:4 Program content and length, and Association, the Education Commission of teachers' perceptions of program impact. the States, the Carnegie Forum on Modify Question 31 by adding "classroom Education and the Economy, and the management skills" and "teaching limited Holmes Group. These reports emphasized English proficient (LEP) students" to the the need to professionalize teaching in five types of program content, deleting the order to improve education and stem what yes/no participation question, and adding was described as "a rising tide of
103
1 1 0 mediocrity." Renewal of a competent and states in turn can award subgrantsto teaching force, as well as recruitment, local areas. Under the Improving America preparation, and licensure werenow Schools Act, the Eisenhower Professional recognized as central to educational reform Development program will support efforts (Green, 1987; Darling-Hammond & sustained long-term IPD efforts relatedto Cobb, 1995). academic standards. In addition, provisions in ESEA for disadvantaged These reports stimulated a number of children and bilingual education include initiatives to establish and enforce funds for professional development. professional standards for teachers: professional organizations such as the Although teacher developmentwas not National Science Teachers Association included in the Governors' six original established standards for certifying goals, it was added in the Goals 2000 Act members, the National Board for in 1994, which renumbered the goals Professional Teaching Standards was making the goal for teacher education and established in 1987 to provide advanced professional development Goal 4. The professional certification of teachers, the goal states: 20 member states of the Interstate New Teachers Assessment and Support By the year 2000, the Consortium (INTASC) developed model Nation's teaching force will licensing standards and assessments for have access to programs for beginning teachers, and the National the continued improvement Council for the Accreditation of Teacher of their professional skills Education is reexamining its standards to and the opportunity to make them consistent with those of acquire the knowledge and INTASC and the National Board (Darling- skills needed to instruct and Hammond & Cobb, 1995). prepare all American students for the next century. In 1990, President Bush and the nation's Governors established the National In 1994, Secretary Richard W. Riley Education Goals and set a target date of established the U.S. Department of the year 2000 for achieving them. This Education's Professional Development constituted a commitment to a nationwide Team to examine research and exemplary effort to reform education around the practices related to professional aspirations of the goals (National development, to guide the Department's Education Goals Panel, 1995a). With the programs and to inform policymakers and advent of the Goals 2000: Educate practitioners across the country. This America Act and the Improving America's team agreed that "the mission of Schools Act of 1994, federal funds became professional development is toprepare and available for improving teaching. Several support educators to help all students provisions of the legislation support IPD achieve to high standards of learning and activities. Under the Goals 2000 development" (USED, no date). legislation, funds for professional development are made available to states,
104
111 To provide assistance in implementing the and connected to classroom practice" legislated activities, the U.S. Department (CPRE, 1995a, p. 10).Professional of Education plans to publish a series of development is one of the two strategies idea books to share effective practices with most frequently used by the states for educators in carrying out reform efforts. changing practice based on the logic that The first of the series, Implementing changing practice requires changing the Schoolwide Projects: An Idea Book for skills, knowledge, and beliefs of classroom Educators was published in 1994.It teachers (CPRE, 1995b, p. 4).(The other includes a section on professional strategy is funding local initiatives and development that provides a number of model schools.) suggestions for IPD as well as descriptions of programs in specific schools. As part of an ongoing effort of the NSF to increase the impact of its Teacher Other federal government agencies also Enhancement Program, the Division of initiated major programs to reform Elementary, Secondary, and Informal education. For example, in 1991 the Education started a project known as the National Science Foundation initiated a Local Systemic Change Through Teacher Statewide Systemic Initiatives Program Enhancement Project (LSC). This project (SSI) to reform science, mathematics, and was started because NSF staff recognized technology education. During the first the need for continuous staff development three years of the program, the Foundation in the schools and the importance of signed cooperative agreements with 26 working with whole schools instead of states to undertake comprehensive reform focusing on individuals if reform is to initiatives in these fields, typically over a happen. The LSC project consists of a set period of five years. The SSI is of district-based projects designed to complemented by analogous programs for reform science, mathematics, and Urban and Rural Systemic Initiatives. technology education through intensive upgrading of their K-8 teacher work force. The SSI programs make heavy demands on In addition to implementing quality teachers."Teachers not only need to curriculum materials, the projects must understand the requirements of the new provide at least 100 hours of professional systems, but in many instances, they are development in content and pedagogy to expected to change their practice, enhance all participating teachers. This program, their subject-matter knowledge, develop which began in 1994, has funded 24 new curricula, and serve as overseer and projects (involving 90 districts of varying assessors in the new process.... They need sizes) for up to five years. opportunities to acquire the necessary knowledge and skills, to practice new State Reform Initiatives strategies, and to interact with other teachers about what works and how to States have also initiated (and continue to solve common problems. In short, a initiate) reforms of teacher education in radically restructured and refocused system connection with their school restructuring of professional development is needed. efforts.In 1988 The Southern Regional The system must be intensive, continuous, Education Board (SREB) surveyed deans
105 112 of education and deans of arts and sciences current professional development. For to determine the changes in the education example Little (1993b) states that "... of teachers since 1981. The resulting states and districts have been relatively report identified education of teachers as a slow to reshape professional development priority in education reform (SREB, in ways that respond to the complexities 1988). and ambiguities of reform." One of the conclusions of CPRE's 1990 Reform Up The Education Commission of the States Close study of high school mathematics (ECS) was also concerned with linking and science in six states was that therewas teacher education to school reform.State "...little by way of staff development that leaders expressed dissatisfaction with appeared up to the challenges ahead. Most current recertification requirements, noting staff development we foundwas that they were heavy on costly inservice fragmented and piecemeal, identified and activities with little to show for the delivered by persons distant from the expenditures (Frazier, 1993). In the classroom, and with little, if any, explicit 1990s, they expect an increase in connection to strengthening academic challenges to the accumulation of random instruction" (Porter et al., 1994).Further, course credits that have little significance in discussing the reform of professional to the teacher or the district.State leaders development Sykes (1996) notes that two stressed that in outcomes-based systems, judgments form the contemporaryconcern teacher IPD, whenever possible, should (1) for the professional development of be related to making a teacher more teachers. The first is that teacher learning effective in helping students meet local and must be the heart of any effort to improve state goals and (2) should be designed to education and the second that conventional benefit the school and school district in professional development is sorely reaching organizational goals. ECS inadequate. He considers that these two recognized the need for continuing judgments represent the most serious education and recertification of teachers by unsolved problems for American education recommending that states "...should today. He notes the ineffectiveness of the require recertification programs related to "one-shot workshop" in changing what individual teacher needs and advancement goes on in schools and classrooms and of school and district needs and objectives" asserts that the resources for IPD "...are (Frazier, 1993). The Commission also too meager and their deployment too noted the potential of the new professional ineffective to matter." Although isolated development schools to provide an efforts are under way to promote teacher opportunity for higher-quality IPD learning that will lead to improved activities than are currently available in practice, wide-scale efforts have yet to most districts emerge. With the many reform initiatives under way and the extensive professional As of July 1995, 49 states and the District development that will take place between of Columbia were engaged in standards- the 1993-94 SASS and the 1998-1999 based education reform (American SASS, it is extremely important that NCES Federation of Teachers, 1995). However, measure change in this activity and its experts are critical of the effectiveness of extent and effects as fully as possible.
106
113 Implications of Reform for IPD thoroughly tested models of skill training with opportunities for classroom practice Little (1993a) has noted that most current and classroom coaching and/or reform initiatives fit into one or more of consultation.Skill development models five streams of reform, all of which can be very effective for training related to present challenges to teachers: specific transferable skills and new ideas.
Reforms in subject-matter teaching Today most IPD is carried out by school (standards, curriculum, and districts.It consists of formal education pedagogy), activities such as workshops, inservice programs lasting a day or a half day at Reforms centered on problems of which experts lecture and that may include equity and the increasing diversity each teacher's choice of workshops led by of the student population, trainers. The programs may provide material or suggestions that are useful to Reforms in the nature, extent, and the teachers, but there is seldom follow-up uses of student assessment, to evaluate the effectiveness and utility of the programs. Typically they have little Reforms in the social organization effect on practice because they lack focus, of schooling, and intensity, follow-up, continuity, and linkage with the district's goals for student Reforms in the professionalization performance. (Corcoran, 1995). Another of teaching. common form of IPD is highly theoretical university coursework; half of all teachers These reforms call for major reported earning college credits during the improvements in students' outcomes period 1988-91 (NEA, 1992). But these including critical thinking (which may not are not the types of IPD that will meet the be part of the teachers' current practice); demands of reform. In discussing the identifying and altering classroom practices condition of teaching in America today, that contribute to student failure; authentic Darling-Hammond (1995) notes that assessment, although teachers may not although attempts are presently under way have the skills to design and implement across the country to make a strategic such assessment; and school restructuring investment in the professional development that may be based on principles rather than of teachers, they are embryonic and practices, without models to translate the scattered rather than systematic. She principles into instructional strategies. recognizes, however, that "... the Most of the existing resources for possibilities for rethinking how schools professional development that are limited structure the use of teacher time, the to skills training are not ready to meet the opportunities for team teaching and demands of these reforms that call for collaboration, the development of teacher expanding teachers' opportunities to learn, and school networks, and the experiment, consult, and evaluate. This responsibilities of teachers are probably does not imply that there is no longer a greater now than they have ever been." role in professional development for the These opportunities constitute some of the
107 114 characteristics of good professional sufficient intensity.(Helps teachers development. The next section considers meet the requirements of reforms this topic more extensively. for deeper knowledge of subject matter.) Reform-Oriented Approaches for Professional Development Professional development (or practice schools)--although Corcoran (1995, pp. 5-6) describesseven primarily used in preservice relatively new approaches to professional development, they could bring development that may be effective in novice and experienced teachers reform and comments (paraphrased) on together with university clinical their desirable characteristics: faculty to improve their practice through observation, low-risk Joint work--shared responsibility experimentation, reflection, and for tasks such as team teaching, coaching. curriculum committees, or other jobs that create independence National board certification--the among teachers and require process of applying for certification cooperation.(Provides is thought to be excellent opportunities for exchange among professional development since it teachers and reflection about requires teachers to document their practice.) practice, reflect on their strengths and weaknesses, and demonstrate Job enrichment--expansion of specific knowledge and skill. teachers' work in ways that require new skills, such as the scoring of Teachers as researchers--research in portfolios in Vermont or serving as classrooms and schools in mentors to beginning teachers. cooperation with their colleagues (Provides opportunities for teachers and university faculty.Frequently to discuss their practice and share directed at problems identified by ideas.) teachers, or may be defined by academic interests.(Benefits: Teacher networks--focus on specific stimulate discussions, help subject-matter and seek to deepen organizations define problems, and teachers' understanding of content lead to changes in practice and and their facility with new teaching policy.) strategies. (Offer access to a "professional community" and These approaches are consistent with the discourse about improving principles of high-quality professional practice.) programs that are discussed in a later section. They also share some common Collaborations between schools and characteristics: colleges--often required to fill need for professional development of
108 115 They respect the expertise of School-University collaborations accomplished teachers targeted at school reform--on the whole, these partnerships have They are integrated with teachers' formed between individual activists work in universities and schools or districts, or between individual They are based on current research consultants and schools, or between on teaching and learning departments of education and local schools. They have not routinely They recognize teachers as a incorporated faculty from subject valuable source of information matter departments. They hold regarding effective professional promise as vehicles for more development and include them in its effective professional development, design and implementation e.g., insider/outsider attached to the school to provide support, expand Little (1993a, pp. 4-5) also addresses access to resources and to critique alternatives to traditional approaches. school progress, e.g. ,The Coalition These alternatives are ones "...that engage of Essential Schools. Other teachers in the pursuit of learning in ways partnerships such as the Chicago that leave a mark on their perspectives and Project on Learning and Teaching their practice." She describes four have the goal of promoting alternative models, the first two being ones breakthroughs in conceptual that were also listed by Corcoran: understanding for the teachers and to immerse them in math Teacher collaboratives and other experiences. networks--subject-specific teacher collaboratives share the view that Subject matter associations--clearly teachers' professional development they are exerting increasingly encompasses (1) teachers' powerful influences in the design of knowledge of academic content, subject curriculum and assessment instruction, and student learning; standards. They are positioned to (2) teachers' access to a broader exert strong influence on teachers' network of professional dispositions toward reform relationships; and (3) teacher proposals. Their effect may be leadership in the reform of multiplied if the association's most systemwide structures. active members also occupy Collaboratives underscore teachers' leadership roles within their school, involvement in the construction of district, or union. subject matter knowledge. Thus they prepare teachers to make Special institutes and centers- informed responses to reforms in teachers say they provide a good subject matter teaching and student professional development assessment. experience. They offer great depth and focus, enough time to grapple
109
1 1 -4- with ideas and materials, thesense Principles of High-Quality of doing real work rather than Professional Development being talked at, and an opportunity to consult with colleagues and Programs experts. (They also cost more per participant, and are less accessible How often do you hear than more modest local programs.) statements to the effect that Teachers enjoy the opportunity for the continuous professional sustained work with ideas, development of teachers is materials, and colleagues. the key to school improvement? ...the general The approaches described above can be endorsement of inservice effective only if the structures exist to education means nothing make them available to teachers and to without an accompanying provide them support for classroom understanding of the implementation of what they have learned. characteristics of effective as The three common characteristics of good compared with ineffective staff development structures are identified inservice education efforts. by Loucks-Horsley et al. (1989, pp. 45- Nothing...has promised so 48): much and has been so frustratingly wasteful as the Support for the practice and thousands of workshops and refinement of new behaviors in the conferences that led to no classroom significant change in practice when the teachers Opportunities for teachers to talk returned to their classrooms and work together to reinforce, (Fullan, 1991). problem solve, and encourage change Although successful school reform requires many ingredients, the one essential A clear message that the new ingredient is the classroom teacher. The behaviors are important and Goal 4 Resource Group of the National teachers are expected to use them Education Goals Panel is well aware that only recently have we fully appreciated the They describe several types of effective ways of teaching complex subject matter to staff development structures for elementary diverse students and consequently many of science:institutes similar to NSF- our current teachers are under- or sponsored institutes of the past, teachers unprepared. They provide a compact centers, and networks and partnerships, all definition of high-quality IPD: of which were mentioned by either "Professional development should be Corcoran or Little. continuous, sustainable, site-based, context driven, focused on student learning and designed to promote school-wide innovation and change" (NEGP, 1995c).
110
117 Corcoran (1995) states that the instructional and assessment reform movement will require a shift from strategies for instilling higher-order a behaviorist approach to teaching competencies, school culture and "...approaches which actively engage shared decision making. students in the construction of knowledge." A number of experts and organizations Model constructivist teaching. have discussed principles and policies for Teachers need opportunities to professional development programs that explore, question, and debate in are consistent with the current reform order to integrate new ideas into efforts.Corcoran (1995) summarizes their their repertoires and their work and provides a list that is based on classroom practice. the work of G. Griffin (1982), B. Joyce and B. Showers (1982), S. Loucks- Offer intellectual, social, and Horsley, C. Harding, M. Arbuckle, L. emotional engagement with ideas, Murray, C. Dubea, and M. Williams materials and colleagues.If teachers (1987), N. L. Zimpher and K. R. Howey are to teach for deep understanding, (1992), J. W. Little (1993), H. Price they must be intellectually engaged (1993), National Staff Development in their disciplines and work Council (1994), and H. Hodges (1994). regularly with others in their field. Their suggestions include programs that incorporate the following principles or Demonstrate respect for teachers as policies: professionals and as adult learners. Professional development should Stimulate and support site-based draw on the expertise of teachers initiatives.Professional and take differing degrees of development is likely to have teacher experience into account. greater impact on practice if it is closely linked to school initiatives Provide for sufficient time and follow- to improve practice. up support for teachers to master new content and strategies and to Support teacher initiatives as well as integrate them into their practice. school or district initiatives.These initiatives could promote the Are accessible and inclusive. professionalization of teaching and Professional development should be may be cost-effective ways to viewed as an integral part of engage more teachers in serious teachers' work rather than as a professional development activities. privilege granted to "favorites" by administrators. Are grounded in knowledge about teaching. Good professional Little (1993a) also discusses principles and development should encompass adds three: expectations educators hold for students, child-development theory, curriculum content and design,
111 118 Should take explicit account of the Promotes continuous inquiry and contexts of teaching and the improvement embedded in the daily experience of teachers life of schools
Should offer support for informed Is planned collaboratively by those dissent who will participate in and facilitate that development Should place classroom practice in the larger contexts of school Requires substantial time and other practice and the educational careers resources of children Is driven by a coherent long-term The U.S. Department of Education's plan Professional Development Team also developed a set of principles (1995). Is evaluated ultimately on the basis Their principles reflect related research of its impact on teacher and exemplary practices and the review effectiveness and student learning; and comments on the principles by a large and this assessment guides number of people and organizations. The subsequent professional team provided ten principles: development efforts
Focuses on teachers as central to Although the Department's list repeats student learning, yet includes all some of those suggested by Corcoran and other members of the school Little, there are no inconsistencies among community the principles.
Focuses on individual, collegial, More recently, Howley and Valli (1996) and organizational improvement proposed another set of principles for effective professional development, which Respects and nurtures the they named "the consensus model of intellectual and leadership capacity professional development. The model is of teachers, principals, and others based on the implications of recent in the school community research on learning for professional development. They first summarize the Reflects best available research and convergence of research on learning practice in teaching, learning, and reported in several recent syntheses of leadership such research. Five "learner-centered principles" of learning have been Enables teacher to develop further identified: expertise in subject content, teaching strategies, uses of One's existing knowledge serves as technologies, and other essential a foundation of all future learning elements in teaching to high standards
112
1J9 The ability to reflect upon and Is primarily school-based and regulate one's thoughts and integral to school operations behaviors is essential to learning and development Provides learning opportunities that relate to individual needs but are, Motivational or affective factors for the most part, organized around along with the motivational collaborative problem solving characteristics of the learning tasks play a significant role in the Is continuous and ongoing, learning process involving follow-up and support for further learning--including support Learning processes through various from sources external to the school common stages of development influenced by both inherited and Incorporates evaluation of multiple experiential/environmental factors sources of information on (a) outcomes for students and (b) Learning is as much a socially processes that are involved in shared undertaking, as it is an implementing the lessons learned individually constructed enterprise through professional development
This research on learning has stimulated a Provides opportunities to engage in number of new studies of professional developing a theoretical development that reach remarkably understanding of the knowledge and consistent conclusions "...with respect to skills to be learned the characteristics of professional development that are most likely to lead to Is integrated with a comprehensive improvements in actions of educators that change process that deals with the contribute to student learning." Based on full range of impediments to and these studies, Hawley and Valli propose facilitators of student learning their new consensus model of professional development with eight design principles: Hawley and Valli's list is further evidence of the consensus among researchers--all Driven, fundamentally, by analyses but one (the fourth) of their principles are of the differences between (a) goals included in the lists considered earlier. and standards for student learning Hawley and Valli not only provide and (b) student performance illustrative references that support each of the principles in their model, but they Involves learners (e.g., teachers) in document the research base for this list by the identification of their learning tabulating the relationships between the needs and, when possible, in the five learning principles and the design development of the learning principles of the consensus opportunity and/or the process to model of professional development. be used
113 120 Use of the Principles to Develop planning and coordination of IPD should SASS Items Related to IPD be added to the Principal Questionnaire. Quality The following suggestions for the itemare drawn from Corcoran's framework for The list of principles is heterogeneous--it reviewing IPD policies and practices contains aspects of the planning and (Corcoran, 1995), from expressed needs of coordination of IPD; ways in which the Goal 4 Resource Group of the National schools organize to facilitate and stimulate Education Goals Panel (1995), and from teacher learning; the growth opportunities the principles published by the U.S. being provided for teachers; school Department of Education's Professional support for professional development; and Development Team. (Questions have been school environment. Each of these topics added that are not in the above list of should be explored in SASS to study the principles and some that are included in extent to which current IPD programs are the list have been reworded.) consistent with the principles of high- quality professional development Item 3:5 Planning and Coordination of programs. IPD. Provide columns for answering yes or no to each question. Information about school-based IPD programs could be obtained by adding Is there a state plan for IPD and questions to the Principal Questionnaire are there state priorities? and Teacher Questionnaire. Teachers Does the state or district require could also provide information about the that schools develop plans? off-site IPD in which they participate. Are IPD activities tied to school improvement? Planning and Coordination of Professional Is there coordination among Development providers of IPD? Are teachers required to develop Two items are suggested related to professional improvement plans? planning and coordination of professional Are teachers involved in the development. The first pertains to development of the learning elements of effective planning and opportunity and/or the process to coordination of IPD. The second to be used? reasons why teachers choose not to Are teacher salary increments participate in IPD. Although the second dependent on the job relatedness of item addresses neither planning nor IPD activities? coordination, it provides information Are state initiatives to set standards needed by policymakers to take and develop curriculum frameworks appropriate corrective actions, actions that and new assessments supported by may include improvement of their planning appropriate professional and coordination of IPD. development? Is your school or school district Elements of planning and coordination of engaged in partnerships that will IPD. An item on effective elements of promote community stakeholders'
114
.121 support of programs for Teaching load too burdensome professional development of Committed to other activities that educators? required my time, thought, and energy Reasons for nonparticipation IPD programs. Not persuaded that participation Little (1993b) discusses the wide variation would make a difference to my in profiles of participation in IPD by students teachers with comparable experience and Discouraged by failures of teaching assignments. She notes that these administrative leadership differences persist even in schools formally Truly discouraged about teaching committed to reform initiatives. She illustrates this point with data from the School Organization for Teacher Learning Illinois Writing Project in which less than and Other Growth Opportunities Provided half the teachers in urban schools attended for Teachers the after-school workshops. Understanding why teachers choose not to participate in An item on the ways schools organize for IPD programs is important to policymakers teacher learning by integrating teacher so they can take appropriate action to development into the daily activities of increase participation. Such information teaching and other growth opportunities could be obtained by adding an item to the provided for teachers should be added to Teacher Questionnaire for teachers who the Principal Questionnaire. The questions have not participated in IPD during the are derived from Corcoran's framework prior year. The options in this item are for reviewing professional development paraphrased from Little's specific policies and practices and the Goal 4 illustration. Resource Group. They also stem from two of the six aspects of school Item 4:6 Reasons for Nonparticipation in organization identified by Little (1996) as IPD Programs. Select up to three reasons related to teachers' learning and why you chose not to participate in IPD professional development:(1) extent of programs during this school year.Enter collective focus on students and shared "1" for the most important reason, if you responsibility for student learning, and (2) select two or three reasons enter "2" for teacher assignment policies and practices the next most important, if you select three that satisfy criteria of fit, stretch, and reasons, enter "3" for the least important community. (The questions include some reason. not in the above list of principles and some that are reworded.) Priority Item 5:7 School Organization for Teacher Unimpressed with the quality of Learning and Other Growth Opportunities the program Provided for Teachers. (Provide columns Already expert in the practices for answering yes or no to each question.) of the program Pressed by the demands of too Are growth opportunities built into many projects teachers' workdays?
115 122 Do teachers have regular Do all teachers have full and equal opportunities to work together? access to high-quality IPD Do teachers have a high level of activities? collective responsibility for student Do these opportunities vary across learning? grade levels? Do teachers engage in systematic, Do the state colleges and sustained, collective study of universities provide appropriate student work--coupled with a courses accessible to all teachers? collective effort to figure out the Does your school have sustained roots of student work in the partnerships with other practice and choices of teaching? organizations (e.g., professional Does school policy support the development schools or professional individual and collaborative associations) that provide a home investigation of selected problems for professional development and questions that arise in teaching? options? Is it school policy to develop the organizational habit of shared Support for IPD student assessment? Are teacher assignments based on In addition to embedding teacher learning making the best use of an in the daily work of teaching and individual teacher's existing providing other opportunities for IPD, knowledge, experience, and schools and districts can provide several interest? types of support for IPD. They can Are teacher assignments based on provide incentives, time, and support for stretching teachers' understanding teachers to participate in IPD programs. and skill as well as using their existing expertise? Incentives to participate in IPD. Are teaching assignments designed Policymakers need to balance individual to configure a staff in ways that and organizational interests in IPD, and to provide a basis for professional provide incentives so they are aligned. exchange, mutual support, or The arrangements for IPD should support shared inquiry? schoolwide improvement and at the same Are teachers performing time stimulate the teacher's professional professional or administrative tasks growth and engagement in teaching, and requiring significant skills? support career advancement (Corcoran, Is support provided for beginning (1995, p. 6). An item should be added to teachers? the Principal Questionnaire on incentives. Does your school district support Again, the options in the question come teachers who are seeking National from Corcoran and Little. Board Certification? How much time is set aside for Item 6:8 Incentives to Participate in IPD. professional development during the What incentives are provided for teachers school year? (Provide three time to participate in professional development options.) and to improve their practice? (Provide
116 123 columns to answer yes or no to each Corcoran (1995) discusses one of the steps question.) policymakers should be taking to improve professional development for teachers- Is professional development linked increasing the time available for teacher to personnel evaluation and interaction and professional development. recertification? He summarizes five approaches described Do districts reimburse college by Watts and Castle (1993) that have been tuition for graduate study? used to increase the time available for o Are salary increments linked to IPD: professional development? o Does professional growth bring Using substitutes or releasing increased responsibility, status, or students. Some schools are recognition? effectively using one morning or Are school resources available for afternoon a week for teacher teachers to participate in development and other professional community and improvement activities. However, personal endeavors beyond the this approach provides only small school? blocks of time and is often resented by parents. It would also be useful to ask the question, Purchasing teacher time by using How do the incentives affect permanent substitutes, retirees, or teachers in different grade levels, giving compensation for weekends or career stages? or summer work. This is expensive, sporadic, and some but this would require a separate item with teachers will not participate on a different structure. weekends or during the summer.
Providing time for professional development. Scheduling time by providing In the Foreword to the publication common planning time for teachers Breaking the Tyranny of Time: Voices working with the same children or from the Goals 2000 Teacher Forum teaching the same grade on a (USED, 1994) the Secretary of Education, regular basis.This is often done in Richard W. Riley, describes the critical schools using instructional teams, element of time as one of the greatest but it could be done in many more issues in education reform. The teachers schools if assistance was provided who participated in the forum, 119 in with block scheduling. number, identified time as the most critical resource for the success of school reform. Restructuring time by permanently It is no surprise that one of the eight altering teaching responsibilities, recommendations of the conference was the teaching schedule, school day, "We recommend that teachers be provided or school calendar. This has with the professional time and serious implications for busing, opportunities they need to do their jobs." union contracts, facilities
117 124 maintenance, state regulations, and Item 7:9 Providing Time for Professional budgets.It also means changing Development. Which of the following public expectations--a reason few actions (a list developed from the five schools or districts have taken this approaches described above) have been approach. taken in your school to increase time available to teachers for IPD. (Columns Making better use of available time should be provided for answering yesor and staff.Decrease the hours no to each action.) teachers spend in the classroom to provide them with more time for Support for IPD in main teaching assignment professional work. Although field.In Question 33 of the 1993-94 costly, the costs could be SASS, the first two questions pertain to minimized by: providing time for IPD and the other questions refer to monetary support or Occasionally substituting support for professional growth credits. appropriate television Although this question appears to overlap programming for regular the proposed Item 7, this question is instruction; limited to IPD related to the teacher's main teaching assignment field and is Using adult volunteers or addressed to teachers rather than to the older students to provide principal, which makes it possible to look extracurricular activities for at equity among groups of teachers in the children; allocation of these types of support. Therefore this question should be repeated, Using occasional large but with the addition of an item for "leaves classes for special topics, or sabbaticals." for exposure to the arts, or presentations of outside Item 8:10 Support for IPD in Main "experts"; Teaching Assignment Field. Repeat Question 33 of the 1993-94 SASS, but Using independent study to starting with the addition of an item for let students pursue projects "sabbaticals and leaves" and a change in on their own: and/or the wording of the current first item to read "other released time from teaching." Involving more students in community service School Environment activities. Although high-quality professional An item should be added to the Principal development programs that influence the Questionnaire asking what actions have knowledge and abilities of teachers are been taken to provide teachers with more important, teachers also need to work in time for professional development based an environment that is supportive of good on the approaches described above. teaching. Research on educational quality, teacher professionalism, policy
118
125 implementation, effective schools, and These three questions should be repeated educational change suggests that several in the 1998-1999 SASS to make it possible characteristics of the school environment to measure the major changes in school are related to effective education (Loucks- environment that can be expected between Horsley et al., 1989): 1993-94 and 1998-1999 due to the systemic reform efforts under way in many Clear purposes and outcomes schools and districts.These efforts can be Adequate, appropriate resources, expected to lead to (1) increased including time, staff, and materials professionalization of teachers and (2) A robust conception of staff schools, classrooms, and teachers that development value questions, experimentation, risk Norms of experimentation, risk taking and collaborative problem solving. taking, collegiality and Both of these results are dependent in part collaboration on school environment. Involvement in decision making Leadership and support Some augmentation of the questions, particularly Question 47, might be The importance of these characteristics of desirable. The report by Mullens et al. school environment were recognized in the (1996) included a review of the items on development of the 1993-94 SASS in three professional development in over 25 of the questions about teachers' surveys. They report finding six surveys perceptions and attitudes toward teaching: that include items on 32 elements thought to affect school culture.It would be Question 44. At this school, how desirable to match these 32 elements much actual influence do you think against those used in the SASS questions teachers have over school policy in to look for possible additions to the SASS each of the following areas? (See questions.It would also be desirable to Appendix A for the list of areas group the statements for each of Loucks- and the scale used in this question Horley's six characteristics so that it and the following question.) would be easier for teachers to understand the purpose of the question. Question 45. At this school, how much control do you feel you have Item 9:11 Teachers' Influence Over IN YOUR CLASSROOM over School Policy.Repetition of SASS each of the following areas of your Question 44. planning and teaching? Item 10:12 Teachers' Control in the Question 47. Do you agree or Classroom of Planning and Teaching. disagree with each of the following Repetition of SASS Question 45. statements? (A list of 25 statements that relate to the six characteristics Item 11:13 Teachers' Perceptions of of school environment listed by School Environment. Repetition of SASS Loucks-Horsley follows.) Question 47 with items grouped by the Loucks-Horsley characteristics and with
119 126 possible additional items identified in attract, recruit, prepare, retrain, research. and support the continued professional development of teachers, administrators, and other Data Needs for the Year 2000 educators, so that there is a highly National Education Goals Report talented work force of professional educators to teach challenging In 1994, the Goals 2000 legislation subject matter. formally authorized the National Education Goals Panel (NEGP), a bipartisan Partnerships will be established, committee of state and federal officials that whenever possible, among local had been meeting since 1990 to monitor educational agencies, institutions of progress toward the goals. Charges to the higher education, parents, and local panel included continuing to play a major labor, business, and professional role in tracking education reform through associations to provide and support its annual reports on progress toward programs for the professional meeting the education goals. The 1995 development of educators. goals report (National Education Goals Panel, 1995b), which is the Panel's fifth The report also provides nine measures of report, includes a section on teacher progress toward the goal of which three education and professional development deal with professional development and that lists the four objectives under goal 4: two with teacher support. The other measures relate to preparation to teach All teachers will have access to limited English proficient students and preservice teacher education and preservice teacher education and continuing professional certification. All of the measures were development activities that will derived from the Teacher Questionnaires provide such teachers with the in the 1991 and 1994 School and Staffing knowledge and skills needed to Surveys. In fact the NCES expanded the teach to an increasingly diverse section on staff development in the 1993- student population with a variety of 1994 SASS to provide information needed educational, social, and health by NEGP. needs. Enhancements for the Year 2000 Goals All teachers will have continuing Report Derived from the Suggested Items opportunities to acquire additional knowledge and skills needed to The items that have been suggested for teach challenging subject matter inclusion in the 1998-1999 SASS would and to use emerging new methods, have a very positive effect on the goals forms of assessment and report for the year 2000 by providing technologies. additional detail for measures used in the 1995 report as well as a number of States and school districts will additional measures. Each of the five create integrated strategies to measures on professional development and
120
127 teacher support in the 1995 report are Support is tabulated for all discussed in turn showing the source of the teachers, and for urban, suburban, information used in the 1995 report and and rural teachers. the effect the suggested items could have (Source: SASS Question 33) for the year 2000 report. Item 8 would add one additional type of The three measures of professional support "sabbaticals and leaves" to the list development are direct measures of the in the 1993-94 SASS. goal: Participation in Different Types Participation in Professional of Professional Development Development Activities on Activities: Percentage of teachers Selected Topics: Percentage of who reported that they participated teachers who reported that they in various activities related to participated in inservice or teaching (workshops or inservice professional development programs programs, college courses, and on various topics (uses of activities sponsored by professional educational technology, methods of associations) since the end of the teaching subject field, in-depth previous school year, 1994. study in subject field, and student Participation is tabulated for all assessment) since the end of the teachers, and for beginning previous school year, 1994. teachers, teachers with four-to-ten Participation is tabulated for all years of experience, and teachers teachers, and for urban, suburban, with more than ten years of and rural teachers. experience. (Source: SASS Question 31) (Source: SASS Question 30)
Item 2 would add two topics to the list in Item 1 would greatly expand the list of Question 31. Both of these topics are types of IPD activities. Of special important in reform: classroom importance, it would include new management skills and preparation to teach approaches for IPD that may be effective limited English proficient (LEP) students. in reform. In addition it would provide It would also provide information on the information on the amount of time spent in teachers' opinions about the impact of the each program and the teachers' total programs. monetary expenditure for IPD.
Support for Professional The two measures that deal with teacher Development: Percentage of support are direct measures of the third teachers who reported that they objective and are closely related to received various types of support inservice professional development: (released time for teaching or scheduled time, travel, per diem Support through Formal Teacher expenses, tuition, and/or fees: and Induction Programs: Percentage professional growth credits). of teachers (by experience
121 128 categories) who reported that Since Item 9 is a repetition of Question during their first year of teaching, 44, it will provide no additional they had participated in a formal information. teacher induction program to help beginning teachers by assigning In addition to the above itemson them to master or mentor teachers, professional development and teacher 1994. Information is provided for support, the NAGB report includes two the following categories of measures that deal with preparation to teachers:all, elementary, teach limited English proficient (LEP) secondary, urban, suburban, and students: (1) percentage of teachers who rural.Information is also provided reported that they have LEP students in on change in participation in their classes and have received trainingto induction programs between 1991 teach LEP students and (2) percentage of and 1994. teachers who reported that they have (Source: SASS 1993-94 Question received training to teach LEP students, 35a and SASS 1990-91 Question 1994. The SASS question on training does 28a) not inquire when the training was received so it is not clear whether the training was As indicated earlier, items about induction preservice or inservice. The suggestion in programs for the 1998-1999 SASS will be Item 2, to add a topic on preparation to the subject of a subsequent paper. teach LEP students to the list of topics of IPD programs in Question 31 of the Teacher Influence over School Teacher Questionnaire, would eliminate Policy: Percentage of teachers who this ambiguity and help measure current reported that teachers in their efforts for the first objective of Goal 4. school have influence over school As mentioned, it would also be possible policy in selected areas for Item 2 to include topics related to (determining the content of teaching other special student populations, inservice programs, establishing which would provide even more curriculum, and setting discipline information relevant to the first objective. policy). Information is provided for the following categories of Additional Measures for the Year 2000 Goals teachers:all, elementary, Report secondary, urban, suburban, and rural.Information is also provided In addition to providing more information on the change between 1991 and directly related to the IPD measures in the 1994 in teacher influence over Goals Report for the year 1995, the school policy for the selected areas. suggestions in other items would provide (Source: SASS 1993-1994 additional measures on the following topics Question 44 and SASS 1990-91 for the goals report for the year 2000: Question 39) Item 3: Planning and Coordination of IPD.
122
129 Item 4: Why Teachers Choose Not to Board certification. The Resource Group Participate in IPD Programs. also suggested indicators that show the number of teachers who are seeking to Item 5: School Organization for Teacher become board certified and the number of Learning and Other Growth Opportunities school districts that are supporting teachers Provided for Teachers. who are seeking National Board certification.Information on the number Item 6: Incentives to Participate in IPD. of schools that are supporting teachers who are seeking board certification could be Item 7: Creating Time for Professional obtained from Item 5 which includes the Development. question "Does your school district support teachers who are seeking National Board The selection of indicators for the 1995 certification?" Information on teachers who Goals Report was limited by data are seeking or who have received National availability. The resource group identified Board certification could be obtained by some future data needs, two of which adding questions to the Teacher's might be filled by SASS 1998-1999: Questionnaire in the section on teacher training. The matter of equity regarding the number of teachers having full Item 12:' Board Certification. Add a access to high-quality professional new question with the two parts: Have development activities should be you received National Board certification? reported. Are you seeking National Board certification? Partnerships that provide and support programs for the In summary, the Year 2000 Goals Report professional development of could provide a much more comprehensive educators should be established. picture of progress toward Goal 4 if the 12 suggested items were adopted. Data for both of these topics could be provided by adding questions to the Principal's Questionnaire. As proposed, Summary Item 3 on Planning and Coordination contains the question:"Is your school or How well do the suggested items respond school district engaged in partnerships that to the 14 introductory questions? Early in will promote community stakeholders' the paper it was noted that it is not feasible support of programs for professional for SASS to collect the data required for development of educators? Item 5 on Questions 6 and 10. However, School Organization for Teacher Learning information about some aspects of these and Other Growth Opportunities Provided questions would be provided by other for Teachers contains the question: "Do suggested items.Teachers' perceptions of all teachers have full and equal access to the effectiveness of the IPD programs high-quality IPD activities? would be known and certainly Item 1 on prevalence of IPD by type and Item 2 on
123 130 program content and length would provide PART II a good idea of what the public sector investment (whatever the amount may be) is purchasing. Question 9 will be covered International Comparisons of IPD in a later paper. Excluding these three for Use of Computers and questions, items have been suggested that Advanced Telecommunications provide data that relate directly to all the Equipment remaining questions except the last four. One of these, number 13, is addressed in Part II discusses the value of international part in Part II.Analysis of the data from comparisons and the value of state and the suggested items can also provide some nation comparisons generally and more information relevant to better ways to specifically with respect to IPD. A invest resources (Question 11). number of international comparative Documentation of the range and quality of studies that have been reported in process IPD in 1998-99 can also be the basis for or in the design stage will provide data on suggesting the _changes needed in IPD to IPD and related topics such as school meet the challenges of the reform organization and environment.15 Part II, movement (Question 14).Finally, it however, addresses only one international should be possible to measure how the study, the IEA Computers in Education characteristics of IPD are changing over Study (CompEd Study), which has time (Question 12), since care was taken extensive information on the professional to preserve the options in the 1993-1994 development of teachers. It is discussed SASS when modifications of questions here to allow ample time for careful were suggested. evaluation of the suggestion made in a later section to incorporate items from the Including all 12 suggested items in SASS CompEd Study in the 1998-1999 SASS. If would greatly expand the section devoted implemented, this suggestion would have a to IPD.It is important, however, to large impact on SASS and should be remember the essential role of the teacher considered in the early stages of the in the reform effort and the importance of development of SASS. Part II therefore providing teachers with the degree of continues with a discussion of the need for professionalization needed in reform. We data on IPD for use of computers and should not forget Sykes' assertion that the advanced telecommunications equipment, a most serious unsolved problems for specific proposal to include IPD items American education today are that teacher from the CompEd Study in SASS, and a learning must be the heart of any effort to description of the benefits of doing so. improve education and that conventional professional development is sorely inadequate (Sykes, 1996). Policymakers Value of International need information to address these Comparisons problems.It follows that inservice professional development should be given The SASS measurements could be made the attention and the space in the 1998- more meaningful and the policy-relevance 1999 SASS that it deserves. of the data could be enhanced by
124
1 3 comparisons with data from other nations. Similarities in Cross National Issues A number of educators have discussed the value of international comparative Most developed countries are facing education information. Bradburn and similar education policy issues.Several Gilford (1990) consider that the most countries are involved in reform efforts important use is to improve understanding and many are faced with the issue of how of our own education system. In the to provide high-quality education to a absence of absolute standards for multicultural student body. These educational systems, comparative common concerns enhance the likelihood information can contribute to setting that we can learn from the actions taken by realistic standards and to monitoring the other countries. Some of the issues relate success of educational systems. They note specifically to IPD. Most countries that the value of comparisons with other states are members of the OECD are deepening or the nation, comparisons that have the inservice teaching opportunities, as are advantage of comparing systems that are Asian countries, e.g., Japan, Taiwan, and broadly similar.International China (Darling-Hammond, 1996). There comparisons, however, expand the range is concern about the limited opportunities of comparison beyond the limits of for advancement and promotion in national experience, and can be helpful not teaching. To address this issue, some only for descriptive purposes but also for countries, e.g., the United States and New monitoring. Plomp (1992) also considers Brunswick, Canada, are taking action to that the most important reason for create a career path that would lead toward international comparisons is to improve highly accomplished practice over the understanding of educational systems and course of a teaching career. Korea has to provide policymakers and educators also recognized the need for a teaching- with information about the range of oriented career continuum (Darling- educational quality among various national Hammond & Cobb, 1995). In Spain and systems. Cross and Stempel (1995) note Portugal inservice training is linked to that the value of international information career advancement (EURYDICE, 1995). is that it provides the opportunity to resolve the failings of our system in a Another issue common to several countries uniquely American way. They urge stems from recognizing the importance of concentrating on the reasons behind the giving teachers greater professional decisions made by different countries authority and responsibility. The United concerning teacher training policies. States, Manitoba and Quebec in Canada, Understanding their motives and and the Republic of Korea have responded expectations will help us decide what will by giving teachers greater professional and will not help us improve primary and autonomy and greater voice in creating secondary education in America. standards for preparation, licensure, and practice.Several European and Asian countries have recognized the significant role of continuous professional development as an important part of
125 132 professionalism (EURYDICE, 1995; We turn now to ways of obtaining IPD Darling-Hammond & Cobb, 1995). data for state and nation comparisonsand international comparisons forone topic: use of computers and advanced Value of State and Nation telecommunications equipment. We first Comparisons consider why such data are importantat this time. The policy relevance of the SASS data could be further enhanced by comparisons of state and nation data. SASS certainly Need for Data on IPD for Use of has the potential to provide state dataon Computers and Advanced IPD. Although the SASS by State Telecommunications Equipment publication (NCES, 1994) includes data about teacher characteristics and their This section describes the rapid growth in preservice preparation, it does not include the use and types of use ofcomputers in IPD data. During the current period of the schools, and the political support for extensive reform and restructuring of the introduction of advanced schools it is important for states to know telecommunications equipment in the about the involvement of the current schools. Although IPD in theuse of teaching staff in the reform effort and how computers and other technologies isan teachers are upgrading their expertise in important aspect of the successful their field and in pedagogy to meet the introduction of the equipment in the demands of reform. If the suggestions in schools, little is known about it.There this paper are implemented, it would be are large gaps in the U.S. system of possible to provide state data on the types teacher training: teachers needmore time and extent of IPD activities, the planning to become conversant with computer and coordination of IPD, school technology, to plan lessons that integrate organization for teacher learning and other the computer in classroom activities, and growth opportunities provided to teachers, to learn about computers (Anderson, support for IPD, and the school 1993). The Comp Ed Study is of special environment.It should also be possible interest because of its careful look at IPD for states to compare some characteristics for use of computers and because of its of their professional development activities finding that U.S. teachers had less with those in other countries ina format opportunity for such IPD than teachers in like that used in the NCES publication, countries whose studentswere more Education in States and Nations (1996). proficient than U.S. students in theuse of Because of the central role that teachers computers. This section concludes with a play in student achievement, states that description of the IPD items in the have demonstrated interest in educational Comp Ed Study. achievement in other countries (frequently for economic reasons) would finduses for such information.
126
133 The Use of Computers in Education President Clinton includes a goal to connect all the nation's school classrooms As we move into the age of cyberspace, (and also various other institutions and there are many unanswered questions organizations) to the "Information about the role that computers and other Superhighway."It remains to be seen how forms of technology can play in education. teachers will use access to the Internet in The percentage of elementary and their teaching. secondary school students who use a computer at school is increasing rapidly: Importance of IPin the Use of Computers in the nine years from 1984 to 1993 the and Advanced Telecommunications percentage doubled, increasing from 28.5 Equipment percent to 59.0 percent (NCES, 1996). Fulton (1996) estimates that there were Teacher training is an important aspect of almost 5 million computers for the introduction of computers in schools instructional use in K-12 schools in 1995 because most of today's teachers did not and that the expenditures on technology use computers when they were in reached $2.4 billion a year. Policymakers elementary and secondary school and many are justly concerned about the effectiveness of them did not receive computer of an investment of this size; they need education as part of their preservice additional data about computers, how they training. On the other hand, it is noted by are used in the schools, and howthey Pelgrum and Plomp (1993) that: improve teaching and learning. "...teachers are ultimately the ones charged with the implementation of Changes in the Use of Computers in the computers in educational practice and Schools therefore education of the educators or teacher training is an important aspect of Changes are rapid in this field. Recent the introduction of computers in schools." new releases (Washington Post,1996) In fact, most of the benefits students will illustrate two such changes. The first derive from using computers depend on describes an experimental program in the extent to which teachers integrate Germantown, Maryland, using the computers in their daily classroom computer as an online algebra instructor to activities.In 1992, however, less than replace a human teacher and a program in half the schools in the United States an Alexandria, Virginia, schoolwhere reported having an introductory computer fourth- and fifth-graders can choose to course available for teachers (Anderson, learn math from a computer or a teacher. 1993, p. 52). American teachers have less The second announces the "Net Day" on opportunity to take inservice computer March 9, 1996, when most of California's courses than do teachers in Austria, 13,000 public and private schools were Germany, and the Netherlands and, as scheduled to be wired for the Internet. might be expected, students in these Television on that day showed both the countries are more computer- President and the Vice-President knowledgeable than American students participating in the wiring! The National (Anderson, 1993). Information Infrastructure proposed by
127 134 More recently, a survey to obtain baseline A self-rating scale about the data on the status of advanced teacher's knowledge about and skill telecommunications in public elementary level in using computers and secondary schools asked about barriers to the school's acquisition of advanced Problems experienced in using telecommunication capabilities.Nearly computers. The list of problems two-thirds of the surveyed schools cited includes three that are relatedto lack of or inadequately trained staff and professional development:(1) lack of teacher awareness regardingways teachers lack knowledge /skills to integrate telecommunications equipment about using computers for into curricula as moderateor major instructional purposes, (2) barriers (NCES, 1995). insufficient training opportunities for teachers, and (3) lack of IPD Items in the Comp Ed Study interest/willingness of teachers in using computers The aim of the first stage of the study, with data collection in 1989was "to obtain Teacher's opinion of training needs information about the current status of the use of computers in education, more Training received--providesa list of specifically within schools, ...foruse in 25 topics covered in training. planning, implementation and evaluation in Information on teacher training is the field of computers in education" andto important because the Comp Ed provide baseline information for measuring Study found that teachers tendto change in stage 2, with data collection in teach the topics covered in their 1992. In addition to obtaining datato own training in the lessons for their measure change, "...stage 2 involved students. assessing effects of school variables, and teacher and teaching variables on student Support for training:availability of outcomes in the domain of computer usage training at school; agencies that in schools (functional computer knowledge provide training support; and skills)" (Pelgrum & Plomp, 1994). availability of and type (full time, The survey included questionnaires for teacher, etc.) of computer principals, school computer coordinators, coordinator in the school; time the and teachers of mathematics, science, computer coordinators spend mother tongue, and computer education in helping teachers use computers (in- the grades 5, 8, and 11. school support) or in trainingor study for themselves The Comp Ed Study includesa number of questions about teachers' professional Percent of teachers using computers development related to the implementation in mathematics, science, English, of computers in educational practice: or computer education
128
135 The following section makes a proposal would create, it could replace the Teacher for using some of the Comp Ed IPD items Demand and Shortage Questionnaire in the (augmented by items related to advanced 1993-94 SASS. The most essential telecommunication) in the 1998-1999 questions from that survey could be added SASS and discusses the benefits of doing to the Principal Questionnaire. so. Benefits of Including IPD Items from Comp Ed in the 1998-1999 SASS Proposal to Measure IPD for Computer Education in the 1998- There are several reasons why it would be useful for the 1998-1999 SASS to include 1999 SASS some of the CompEd stage 2 questions about inservice development of teachers. It is proposed that SASS include IPD First, because many types of experts were questions from the Comp Ed Study. This involved in developing the IEA survey, it would require additional questions in the has led not only to interesting findings Principal Questionnaire, a new Computer about the status of professional Coordinator Questionnaire, and either development for computer education, and additional questions on the Teacher identification of large differences between Questionnaire or a separate questionnaire countries in IPD, but has also provided for a sample of teachers in the fifth, data useful to policymakers. For example, eighth, and eleventh grades. data from the Comp Ed Study (a) provided the basis for recommendations concerning Although the Comp Ed Study did not the training needs of teachers, (b) made it address the IPD aspects of teachers' use of possible to identify the relative position of advanced telecommunications equipment, it a country with respect to the availability of would be useful to include questions in the training and support for teachers, (c) Teacher Questionnaire on how teachers use provided a measure of the extent to which networking and other forms of advanced the computer was integrated in classroom telecommunication, the training teachers teaching, and (d) made it possible to receive to prepare them to use technologies determine the relationship of teacher as teaching tools and resources, andtheir training to actual classroom use of awareness of the resources technology can computers. Second, including CompEd offer them as professionals in carrying out IPD questions in SASS 1998-1999 would many of the activities of their jobs(Fulton, make it possible to measure change in the 1996). Fulton develops this topic more amount and character of computer IPD in extensively. A focus group to address the United States from 1992 to 1998. ways to incorporate such questions in the Third, it would permit states to compare survey without losing comparability with their IPD in 1998-1999 with that of other the data from the Comp Ed Study could be states and the nation. And fourth, useful. although there would be six years difference in the data, states could The magnitude of the impact of this compare IPD for their teachers with that of proposal on SASS is recognized. To teachers in other nations at an earlier time. compensate for the response burden it
129 136 In summary, the importance of this Consortium for Policy Researchin proposal is supported by the combination Education. (1995b, May).Reforming of rapid growth in the use of computers science, mathematics, andtechnology and advanced telecommunications education: NSF's State SystemicInitia- technology in the schools, the essential tives (CPRE Policy Briefs, RB-15).New role that teachers play in their effective Brunswick, NJ: Rutgers University, use, the inadequate training in theiruse Consortium for Policy Research in that is available to teachers, and the Education. national will for U.S. studentsto match the achievement of students in other Corcoran, T. B. (1995, June). Helping countries in the use of these technologies. teachers teach well: Transforming Our national leaders have already professional development (CPRE Policy recognized and recently underscored the Briefs, RB-16). New Brunswick,NJ: importance of such technologies in Rutgers University, Consortium forPolicy education. Research in Education.
Darling-Hammond, L. (1995, Summer). References The condition of teaching in America today: Resources for restructuring.New Anderson, R. E.(Ed.).(1993). York: National Center for Restructuring Computers in American schools--1992: Education, Schools, and Teaching, An overview. Minneapolis, MN: Teachers College, Columbia University. University of Minnesota. Darling-Hammond, L. (1996). The Bradburn, N. M., & Gilford, D. M. current status of teaching and teacher (Eds.).(1990). A framework and development in the United States. Paper principles for international comparative presented to the AERA Invitational studies in education. Washington, DC: Conference on Teacher Development and National Academy Press. School Reform, Washington, DC.
Consortium for Policy Research in Darling-Hammond, L., & Berry, B. Education. (1995a, July). Tracking (1988). The evolution of teacher policy. student achievement in science and Santa Monica, CA: Center for the Study mathematics: The promise of state of the Teaching Profession, RAND. assessment programs (CPRE Policy Briefs, RB-17). New Brunswick, NJ: Rutgers Darling-Hammond, L., & Cobb, V. L. University, Consortium for Policy (1995). The teaching profession and Research in Education. teacher education in the United States. In L. Darling-Hammond & V. L. Cobb, (Eds.), Teacher preparation and professional development in APEC members (pp. 221-40). Washington, DC: U.S. Department of Education.
130 1 3 7 EURYDICE, The Information Network in Hawley, W., & Valli, L.(1996). The the European Union and the EFTA/EEA essentials of effective professional Countries.(1995). In Service training of development: A new consensus. Paper teachers in the European Union and the presented to the AERA Invitational EFTA/EEA countries.Brussels: Author. Conference on Teacher Development and School Reform, Washington, DC. Freezer, C. (1993). A shared vision: Policy recommendations for linking Lieberman, A., & Grolnick, M. (1996). teacher education to school reform. Networks, reform and the professional Denver, CO: Education Commission of development of teachers. Paper presented the States. to the AERA Invitational Conference on Teacher Development and School Reform, Fullan, M. (1991). The new meaning of Washington, DC. educational change. New York: Teachers College Press. Little, J.(1993a, October). Teachers' professional development and education Fulton, K. (1996). Technology for K-12 reform (CPRE Policy Briefs RB-11). New education: Asking the right questions. Brunswick, NJ: Rutgers, The State The Schools and Staffing Survey: University of New Jersey. Recommendations for the future (NCES 97-587). U.S. Department of Education, Little, J.(1993b). Teachers' professional National Center for Education Statistics. development in a climate of education reform. Educational Evaluation and Policy Gall, M., & Vojtek, R.(1994). Planning Analysis, 15(2), 129-151. for effective staff development: Six research-based models. Eugene, OR: Little, J.(1996). Organizing schools for University of Oregon, ERIC Clearing teacher learning. Paper presented to the House on Education Management. AERA Invitational Conference on Teacher Development and School Reform, Gandal, M. (1995). Making standards Washington, DC. matter: A fifty-state progress report on efforts to raise academic standards. Loucks-Horsley, S., Carlson M., Brink, Washington, DC: American Federation of L., Horwitz P., Marsh, D., Pratt, H., Teachers. Roy, K., & Worth, K. (1989). Developing and supporting teachers for Green, J.(1987). The next wave: A elementary school science education. synopsis of recent education reform Andover, MA: The National Center for reports.Denver, CO: Education Improving Science Education. Commission of the States.
131 138 Mandel, D. (1995). Teacher education, National Center for Education Statistics training and staff development: (1996). Student use of computers. Implications for nationalsurveys. In G. Indicator of the month. December 1995 Hoachlander (Ed.), From data to (NCES 96-792). Washington DC: U.S. information: New directions for the Department of Education, Office of National Center for Education Statistics Educational Research and Improvement. (NCES 96-901). Washington, DC: U.S. Department of Education, National Center National Commission on Excellence in for Education Statistics. Education. (1983). A nation at risk: The imperative for educational reform (Report Mullens, J.(1995). Classroom to the Nation and the Secretary of instructional processes: A review of Education). Washington DC: U.S. existing measurement approaches and their Department of Education. applicability for the Teacher Followup Survey (NCES 95-15). Washington, DC: National Education Association. (1992). U.S. Department of Education, National Status of the American public school Center for Education Statistics. teacher, 1990-1991. Washington, DC: Author. Mullens, J., Laguarda, K., Leighton, M., O'Brien, E., Wimberly, G., & Murphy, National Education Goals Panel.(1995a). D. (1996). Student learning, teaching The National Education Goals report: quality, and professional development: Executive summary: Improving education Theoretical linkages, current measurement, through family-school-community and recommendations for future data partnerships. Washington, DC: Author. collection. Washington, DC: Policy Studies Associates. National Education Goals Panel.(1995b). Data volume for the National Education National Center for Education Statistics. Goals report. Volume one: National data. (1994). SASS by state, 1990-91 Schools Washington, DC: Author. and Staffing Survey: Selected state results (NCES 94-343). Washington, DC: U.S. National Education Goals Panel.(1995c). Department of Education, Office of Report of the Goal 4 Resource Group: Educational Research and Improvement. Teacher education and professional development. Washington, DC: Author. National Center for Education Statistics. (1995). Advanced telecommunications in Odden, A., & Marsh, D. (1988). How U.S. public schools, K-12 (NCES 95-731). comprehensive reform legislationcan Washington DC: U.S. Department of improve secondary schools.Phi Delta Education, Office of Educational Research Kappan, 69, 593-598. and Improvement.
132
139 Pelgrum, W., & Plomp, T.(Eds.). Sparks, D. (1995). A paradigm shift in (1993).Schools, teachers, students and staff development. The ERIC Review, computers: A cross-national perspective. 3(3), pp. 2-4. Preliminary report, IEA-comped study stage 2.Enschede, The Netherlands: Sparks, D., & Loucks-Horsley, S. University of Twente. (1990). Models of staff development. In W. Houston, M. Haberman, & S. Sikula, Pelgrum, W., & Plomp, T.(Eds.). (Eds.), Handbook of research on teacher (1994). The IEA study of computers in education (pp 264-89). Los Angeles, CA: education: Implementation of an University of California. innovation in 21 education systems. New York: Pergamon Press. Sykes, G. (1996). Reform of and as professional development. Phi Delta Perron, M. (1991). Vers un continuum Kappan, 77(7), 465-467. de formation des enseignants: elements d'analyse. Recherche et Formation (Paris, U.S. Department of Education. (no date). INRP), 10, 137-152. Building bridges: The mission and principles of professional development Phelps, R., Smith, T., & Alsalam, N. (pamphlet). Washington, DC: Author. (1996). Education in states and nations. Washington, DC: U.S. Department of U.S. Department of Education. (1994, Education, National Center for Education November). Breaking the tyranny of time: Statistics. Voices from the Goals 2000 Teacher Forum. Washington, DC: Author. Plomp, T.(1992). Conceptualizing a comparative educational research Watts, G., & Castle, S.(1993). The time framework. Prospects, XXII, (3), 278- dilemma in school restructuring. Phi 288. Delta Kappan, 75, 306-310.
Porter, A., Kirst, M., Osthoff, E., Smithson, J., & Schneider, S.(1994, September). Reform of high school mathematics and science and opportunity to learn (CPRE Policy Briefs RB-13). Madison, WI: University of Wisconsin at Madison.
Shulman, L.(1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4-14.
133 140 1. Mandel (1995) is the source for Questions 4, 7, and10.
2. Because networks have become an increasinglyimportant form of IPD in the reform ofU.S. education, Lieberman and Grolnick (1996) studied 16 networks to ascertain how they are formed, their focus, andhow they are sustained. They note that networksprovide ways of learning thatare more in keeping with the professional lives of teachers. Networks engage "...school-based educators in directing theirown learning, allowing them to side-step the limitations of institutional roles, hierarchies and geographic locations,and encouraging them to work together withmany different kinds of people."
3. Provides information related to introductory Questions3, 4, 7, and 12.
4. Provides information related to introductory Questions4 and 5.
5. Provides information related to introductory Question1.
6. Provides information related to introductory Question8.
7. Provides information related to introductory Question8.
8. Provides information related to introductory Question8.
9. Provides information related to introductory Question8.
10. Provides information related to introductory Question8.
11. Provides information related to introductory Question2.
12. Provides information related to introductory Question2.
13. Provides information related to introductory Question2.
14. Provides information related to introductory Questions3, 4, and 8.
15. The potential of all of these studiesas a source of IPD-related items for the 1998-1999 SASS will be explored in a later paper.The paper will also provide analysis plans describinghow data from these studies might be used in international comparisonsor in state and nation comparisons of IPD--if the 1998-99 SASS includes comparable items.
134 141 DISTRICT LEVEL DATA IN THE SCHOOLS AND STAFFING SURVEY
J. Michael Ross, National Center for Education Statistics
The importance of district-level data in demographic data from the School District systematic assessments of changes in the Data Book), staffing data, and fiscal data. organizational structure of schools as Through an examination of data on magnet educational institutions is increasing.' schools, the feasibility of a multi-level This raises the question of whether the linked approach will be examined in the next Schools and Staffing Survey (SASS) context of a different sampling strategy. should shift its focus toward district-level For some, the need for more processes rather than toward classroom programmatic information that can be instructional models as has been proposed provided by district- and school-level in other papers in this seminar series.' administrators is still an open-ended The arguments for a reconsideration of the question for 1998. For others, however, role of district-level data in SASS are top-down models of analysis are the derived from several sources: (1) a review prevailing, if not the only, strategy to of sociological theories as applied to the study reform implementation effects. organization of education in the United Research studies suggest that the addition States; (2) a critical review of the 1993 of critical reform data at the district level SASS district-level survey and its ability to could enable SASS surveys to become the uniquely answer important research established baseline survey for a large questions; (3) the increasing importance of variety of hierarchical studies by the U.S. "choice" mechanisms for student Department of Education and the National assignment policies in public schools;3 and Science Foundation. (4) recent policy research based upon studies of state-wide systemic reform Administrative Structures and District efforts. Organization
Many of the important organizational Long-term historical trends. The state role issues outlined below are amenable to in education has continued to expand more systematic empirical exploration even rather than diminish during the 1990s. with the 1993 SASS (particularly as they State funding and programmatic control of involve between-state and between-district education has complex organizational variations). The argument presented here implications for school districts and the is that the local school district is still an management of schools within these important mediating organization in the districts.' Through the 1980s implementation of educational policy. relationships among different levels of From this review, it should be apparent government (federal, state, and local), that the 1993 SASS district-level survey became increasingly complex, and at the should be supplemented by more yearly same time, a more layered, formalized Common Core of Data (CCD) school structure of control (multi-level and enrollment information (including 1990 centralized) continued to develop. As the
135 142 external environment imposed a schools as new instructional policiesare multiplicity of new requirements, filtered through fragmented and administrative complexity expanded heterogeneous organizations. substantially at the intermediate (i.e., school-district) level.' The diminishing role of localrevenue in the operation of local schoolsmeans these A major factor in this increased authorities are now held more accountable complexity has been an increase in to standards imposed by external funding categorical funding at the federal level for sources and to the parallel need to a large number of special programs, and centralize the budgeting process and the emergence of new administrative personnel decisions at the district level.' subunits to monitor and oversee these The resulting interdependence between the programs within states and local school district and its component schools has districts.6 These additional layers ofnew required more administrative coordination authority typically have not displaced and an increasing number of professional existing structures, and the current system administrators to "manage" the schools. of governance has preserved the legal One critical element of this administrative autonomy of lower levels of power, growth has been the addition of categorical primarily local school districts. programs (and the external accountability requirements) from state and federal Local districts are sources to each district's operating budget. fundamental governance At the individual school level in larger agencies, by tradition and school districts, the administration of these practice....and their programs has involved a parallel increase influence is extraordinary in in the number of administrators and world perspective. Despite program specialists.In some schools, the recent growth of state however, the bureaucratic burden ofmany and national power, these separate programs has generated a variety districts make a great range of school-wide reforms, and, consequently, of decisions, including those ongoing decentralization efforts have been that bear on levels of designed to counter the organizational funding, the nature of effects imposed by the demands from these educational programs, and external authorities. the teachers to be hired. (Cohen & Spillane, 1992) Analytic Role for SASS District-Level Data By most accounts, the resulting structures are highly fragmented, and from more The primary rationale for the district-level critical perspectives "incoherent" (Cohen, survey (still identified as Teacher Demand 1995).7 The difficult (or unsettled) and Shortage Questionnaire--TDSQ) in its question is whether local responses to first administration in 1987was national these new reform initiatives have amplified concern over the prospects of teacher differences (i.e., increased rather than shortages, particularly in specialized fields decreased variability) between districts and and special programs.' Beforea more
136 143 detailed review of other issues related to from CCD may be more necessary to district-level information is undertaken, it identify these long-term trends. is important to identify the main questions that were the primary focus of the Local Only a few tables in the SASS Statistical Education Agency (LEA) Questionnaire, Profiles contain data from the TDSQ. and how analysis of administrator and Many of the demand issues are adequately teacher surveys provided alternative addressed by results at the school level strategies to satisfactorily answer these (Table 7.2, Statistical Profiles: 1990)11 questions.' In the 1987 and 1990 SASS, and by questions related to the relative the "number of positions filled" was difficulty in filling vacancies by specific consistently high, approximately 99 fields (Table 7.3 and Table 7.4). percent in both years. As reported by Likewise, the analysis of school-level district administrators, less than 0.5 turnover rates (as measured by the percent were vacant or unfilled.Similarly, percentage of teachers who left positions in districts reported that nearly 10 percent of the school in the prior 12 months) allows their teachers were new hires, indicating statistical analysis of school characteristics, that when positions became available they private school status, district-level salary could find qualified teachers from and benefits, and even workplace climate available sources of new college graduates, (Ingersoll, 1996). In regard to the teachers in other districts, private schools, availability of new teachers and other or other sources. characteristics of the teaching profession, the individual teacher survey has provided At the other end of the spectrum, districts more detailed information on new teachers provided counts on how many teachers had (Rollefson & Broughman, 1995).12 And been "laid off" for budgetary reasons (i.e., finally, the teacher followup survey RIFs). In 1990, the percentage was only provides even more detail on the flow (and 0.6 percent. Although the percentage of the reasons) that teachers move to other "laid-off" teachers is not included in the positions or leave the profession (Bobbitt 1993-94 SASS Statistical Profile (because et al., 1991, Bobbitt et al., 1994). previous year estimates of faculty were not asked), a slightly different calculation for Districts after schools. Schools are districts with more than 100 teachers sampled first, and districts are included if indicated that 162 districts had reductions one of their schools is selected. As a greater than 5 percent. Included in this result, the average district in 1993 was group are several well-known districts represented by 1.7 schools. When national where these large cuts merited notice in and state-level estimates are made for local newspapers (and even in Education student enrollment and staffing data, there Week).Clearly, the down-side in teacher is substantial overlap, and consequently the staffing numbers is concentrated in one school-level information and the district year when a budgetary crisis (often data are redundant." Incorporating some precipitated by declining student CCD district data directly into the enrollments) occurs. As the frequency of interview instrument (but taking into SASS shifts to every five years, the consideration a lag between sample inclusion of faculty trends for prior years selection of schools' and data collection
137 144 time frames), and then the refinement of already core variables and other questions this information throughmore structured have been asked for two administrationsof survey questions would be a major design SASS (Levine & Christenson, improvement. In addition, basic data could forthcoming). A large number of be added on enrollment and teacher data personnel and student assignment decisions for a fixed number of prior years." The are made by district-level administrators district-level data should also include (public schools only). But, the personnel aggregated counts (students and teachers) office performs many other critical for all schools in the district. functions for teachers working ina district. Teachers are typically hired by the In the process of linking schools to district and then assigned to specific districts (and both units back to teachers), schools.' Likewise, between-school certain varieties of governance structures transfers of students and teachersare become evident. Some of these state- required to adjust for shifts in student specific categories are more accurately populations, and periodically school identified in the current CCD district openings and closings necessitateeven classification system.' District and larger adjustments. Moreprocess school eligibility criteria should be questions seem to elicit more useful reviewed in order to consider other types information about the outcomes of of instructional and support staff counts administrative decisions (e.g., the school that are included in the agency universe questionnaire could ask how vacancies survey since in some districts they are were filled in a school with a check list becoming more important elements in the and a rating of difficulty).District "reform agenda." administrators could be asked how they have recruited new teachers over the last In reconsidering the utility of the district few years (types of strategies suchas survey, the functions administrators visiting local college campuses, nationalor perform should be reviewed in order to local advertising), in what disciplines consider whether these administrative teachers were hardest to find and then hire responsibilities should be incorporated into teachers for (at this point pay incentives the next SASS. District staffs historically would be relevant to ask).Certification have had limited authority for instruction requirements for new teachersare usually conducted in the classroom by teachers. established at the state level, and district Nevertheless, a high percentage of administrators can offer more information expenditures are no longer associated with about recent changes in these policies.' instructional staff as conventionally defined.' A large number of routine New policies for "student performance" administrative and budgetary tasks (some (e.g., the number of courses required for are generally not relevant to the objectives graduation and more rigorous standardized of SASS) are still performed by district tests) have been enacted in recentyears by staff. Some information, suchas starting many, but not all, states. The district teacher compensation with different questionnaire could ask whethera change degrees and benefits (Table 5.3 and Table has occurred (there should bea high 5.4, 1990 SASS Statistical Profile)are degree of consistency within states) and
138
145 then the respondent would indicate how adjustments in surrounding schools).In the number of course credits changed physically large districts, transportation when the policy was implemented. imposed another set of fiscal and resource Likewise, most large districts have some constraints.In the last 25 years, written discipline and substance abuse fundamental change has slowly displaced policies (primarily for legal reasons).In "the neighborhood school" linked solely to response to federal and state initiatives, residence. In large central city districts, new policies have been adopted, and the the change was abrupt when federal discipline implications for similar desegregation plans imposed new infractions represent a new policy geographic configurations, but the dimension. transition was also facilitated by experimentation with district-wide "magnet Many important policy decisions are based schools" based upon distinct instructional on school board actions, state legislation, programs that would attract different-race and new federal programs that often are students. Besides the obvious benefit of not best analyzed at the individual school dismantling "racially identifiable" schools, level.' For many programs, the funds magnet schools enabled some schools are identified separately in terms of dollar within a district to formulate their own amounts and funding sources. For content emphasis, special themes, or example, the Eisenhower professional school philosophy (and also recruit their development grants are administered by own faculties for these purposes). The local districts after they apply to state traditional uniformity of schools imposed departments for approval of programs. by a central "bureaucracy" no longer Individual districts have wide latitude in maintained its total control over students, the use of these funds and may use them faculty, and instruction in these schools, for such activities as professional and for the first time "market workshops at a local university, national mechanisms" were incorporated in the conferences, or instructional sessions for school selection process (parents had an teachers within a district.' option to choose a magnet for their children or could leave if they were not Even more relevant are student assignment satisfied). At this point, it is evident that issues that are a policy realm under the identification of specific magnet schools nearly exclusive control of school district can only be obtained at the district level authorities rather than individual schools where student assignment policies are (with the exception of some districts still implemented. under federal desegregation court orders). In a narrow historical perspective, the Multi-level analysis. The utilization of district activities in this realm were quite district-level information in prior SASS conventional: fixing physical boundaries surveys and reports has been quite limited, (that rarely changed), constructing a new and the additional questions included for school when enrollment expanded rapidly, the first time in 1993 (AIR, 1996) selecting which school to close when probably will not change interest in enrollments declined, and then deciding complex multi-level analysis.Ingersoll's which schools to consolidate (with limited An Agenda for Research on Teachers and
139 146 Schools (1995) does not identify a single variation in teacher salaries (above and issue where district-level data are a beyond the combined effects of individual decisive factor in an important research teacher background factors and school- question. Only the recent studies by level factors). Using this method,one Chambers (1995 and 1996) explore the could determine if there wasa district- differential effects of school and district specific effect for pay incentives offered to characteristics (i.e., measured by cost mathematics teachers for example, factors that local decision makers cannot controlling for their education background control) on teacher salaries.22 For and years of teaching experience. example, the relationship between salaries Likewise, this method of multiple-level and the racial composition of the district analysis could determine the additional showed that only the percentage of contribution of district policies to students who were Asian Americans had a differential teacher salaries .24 significant effect (the school-level analysis had significant effects only for percentage Multi-level analysis is complex, and this of Hispanic). In the Chambers study, alone might account for the limiteduse of however, the inclusion of district-level district-level data by researchers. (and school variables) resulted in a Similarly, missing data problems substantial loss of schools (17.1 percent) unexpectedly escalate when a district and also teachers in the sample. The nonresponse for a large district eliminates Ingersoll (1996) analysis also had a large several schools from the sample. erosion in its school sample size when Moreover, the original CCD identification such district-level variables as availability is difficult to reconstruct "after the fact" of merit pay plan, paid benefits, and for schools with a missing district survey. district size were included in his analysis Most statistical software packages do not (and most not significantly) of net teacher allow other sources of district data to be turnover rates .23 easily incorporated after SASS analysis files have been merged. Finally, multi- The Chambers study (1996), although level analysis of between-school based upon 1990-91 SASS, does at least differences (controlling for district context) provide a model for making decisions is severely limited by the nonhierarchical about which type of questions should be design feature of the SASS sampling asked to whom based upon multi-level strategy given the small percentage of statistical analysis. In his analysis, only a districts with more than one school per few district-level variables have a district. significant effect in explaining differential teacher salaries.In fact, the three district- Schools after districts. The limited level variables in his regression equations number of key variables in the SASS (district size, racial composition of the district-level questionnaire imposes district, and enrollment growth) were practical difficulties in linking different obtained from CCD, and it is reasonable to levels. How conceptual issues related to assume that data from the district survey district policies in turn impact schools on pay incentives or fringe benefits within each district suggests a different probably would not explain additional design strategy for the new SASS:
140 147 sampling districts first (based upon the each district through the LEA number of teachers in the district's questionnaire. Federal and state program schools), and then sampling schools within funds are allocated to specific schools these selected districts. A larger number of within a district, and accordingly the schools per district' thereby would be number of instructional staff allocated to sampled in districts that have more than these programs (such as magnet schools) 10,000 students, for example. In would be enhanced through this type of Appendix I, a comparison between the multi-level design. average number of schools per district with a district sampling strategy is District-wide Assignment and Choice presented with the results of the 1993 SASS. Districts have developed schools with special academic programs that attract Take a state such as Florida with large students on a district-wide basis in order to county-wide districts. SASS 1993 samples comply with federal desegregation court 258 schools, but they are scattered across orders. These "magnet" schools first 55 districts, giving an average of only 4.7 emerged in the late 1970s in several large schools per district.' With districts northern districts.Initially, they often sampled first, only 20 districts would be were part of a more global restructuring of selected with an average of 20 schools per school attendance boundaries and the district.Utah, a more typical state, would emergence of noncontiguous assignment have an average of 12.9 schools per policies that have been responsible for district drawn from a sample of 20 major revisions in conventional districts (in 1993, the SASS average is 5.5 "neighborhood" attendance zone practices schools per district).Obviously in rural in many districts.Their growth has been states, such as Iowa, an average of 2.8 accelerated by federal grant programs schools per district is not a substantial (Emergency School Aid Act, Magnet improvement over 1.3 schools per district, School Assistance Program-1983), and and fewer sampled districts (67 versus 128 increasing acceptance of voluntary student in 1993 SASS) does not improve district assignment components in desegregation estimates when most districts are quite plans by federal courts in the mid-1980s. small. There are more than 175 districts In the last five years, more districts have that have student populations greater than been released from court supervision and 25,000 students (most have more than 25 have adopted expanded student choice schools) and at least five to ten schools options as a replacement for mandatory from each district would be selected with policies.Other reform movements have this strategy .27 The number of schools, stimulated a broad interest in specialized however, is also a function of the relative "choice" schools in districts (often concentration of students in larger encompassing an entire district with districts, and the average number of "racial balance" a minimal consideration). schools would vary by state.28 The particular combination of conventional Furthermore, a district-level survey would attendance zone and district-wide choice allow direct links to individual schools in policies reflect a district's long
141 desegregation history. In broad terms,one district shifts from mandatoryto voluntary needs to know when a district first assignments for some students). Evenone desegregated its schools; whether a partial magnet school with district-wide or district-wide remedy was required; enrollment options has an impacton which whether a plan was phased inover time; students are enrolled in other schools. the statistical guidelines used for these These attendance policies are administered desegregation plans; and the racial by district administrators and school-level composition (and size) of the district when principals are not fully aware of the the first substantial desegregation planwas interrelationship between who attends their implemented.29 Certain historical school and the schools other students parameters, although difficult to establish attend as a consequence. initially, facilitate the tracking of these periodic modifications that are used by Choice in the 1993 SASS. Questions many large districts to maintain certain concerning magnet schools were asked in levels of "racial balance" through the both the SASS district-level and school- provision of choice policies.' The level questionnaires, and a reanalysis of evolution of district plans also shows the the data offers a preliminary view of the variations in magnet schools, from the inherent difficulties in translating policy "ideal" district-wide option for both options into clean simple survey language. minority and majority students, to schools Alternative question formats andmore that have district-wide options for students reliable results are fortunately available of one race and neighborhood options for from a survey conducted in 1991-92 by students of another race, and to smaller AIR on magnet schools and desegregation programs-within-schools where district- plans. The 1993 SASS district-level wide options are mixed with neighborhood survey asked whether students could assignment policies. In most cases, only "enroll in another school or district outside the recent history is relevant since large their attendance areas"' and if the changes in student assignments require answer was "yes," the respondent could well-publicized announcements by district check enrollment in a magnet schoolor administrators. "enrollment in any school in this district" (and then the respondent would estimate Most large districts are expanding choice the number of students in each program). elements in many schools and the Approximately 579 districts indicated that implementation of more specialized some students attended magnet schools.32 curriculums has an indirect effect on those It would have been preferable if the SASS schools that do not offer "new" programs. questionnaire had provided a definition of Students are selectively drawn from large "choice" that included some reference toa geographic areas and some teachers are special emphasis or distinct curricular "redeployed" from other schools to staff theme, and a district-wide enrollment these new programs. Rarely has a magnet option for some students (rather than school been opened without consideration "outside their attendance areas," which of district-wide racial balance (either to could refer either to school or district attract other-race students for the first time lines).Likewise, obtaining the number of or to maintain desegregation when a magnet schools and their names is feasible,
142
149 even in large districts. Brochures surveys.Certainly in some districts, it is describing these programs are routinely likely that no magnet school would be sent to parents. selected when a district indicates it has magnet schools (maybe one in four schools In SASS, the school was asked two distinct are magnet schools in a typical district questions concerning special or magnet with more than 10,000 students), and only programs.First, it was asked "what type 6 percent of the districts fit this pattern.If of school it is": a "regular" school, or slightly more refined questions had been whether it was a school with a "special elicited from the district administrator program emphasis" such as (e.g., asking for the subset of magnet "science/math" or "performing arts" (or school names), some confusion would have vocational/technical or alternative). The been minimized. More problematic is the second response category should situation when a school identifies itself as correspond to a "total" school magnet a magnet, and the district indicates it has (without distinguishing between dedicated no magnet schools.(In some of these and partial attendance zones). The cases, several magnet schools were percentage estimate from the 1993 SASS is sampled, and the districts are known to 3 percent magnet using the first definition, have comprehensive "choice"plans.34) which is higher by 1.1 percent than the Nonetheless, we were able to match 501 AIR figure. However, the racial districts where there was a "yes" response composition of the magnet schools in both to the magnet school question in the SASS parallels the AIR survey (57.9 LEA survey and by one school percent of the students are in schools administrator in that district (the weighted greater than 50 percent minority in the average number of magnets in large and former versus 56.5 percent in the AIR mid-sized cities was 4.3 schools). Most survey).Likewise, the SASS survey districts did not, however, provide locates nearly 60 percent of the magnets in estimates as to the number of students in central city districts as expected. the magnet schools (They would need a list of the schools themselves to count A second question in SASS asked whether participants.35). the school offered a "magnet program." This could reference "programs-within- This preliminary exercise demonstrates the schools" in contrast to total school potential for SASS to explore complex magnets, but for some schools a "no" policy issues in certainly a more cost- answer to this question could filter out efficient manner than large-scale schools that did not have a curricular focus retrospective surveys. AIR collected most and should not be considered as magnets of its information on magnet schools (in in the conventional sense. Assuming the contrast to general desegregation latter possibility, the SASS estimate is now information obtained in the initial 1.8 percent (1,466 magnet/special schools interview) from a follow-up survey to the weighted"). 127 districts that had choice plans. Another phase of the AIR study involved We now turn back and ask the degree of districts that had received federal Magnet consistency between the district and school School Assistance Plans grants over
143 150 several years between 1985 and 1993. baseline survey for ongoing comparative The grants cover a three-year timespan, state-level studies.First, it is hoped that and 117 districts had obtained a leastone some program (and/or discipline specific) grant. The survey gives some insight into data can be successfully incorporated from the administrative infrastructure that CCD aggregate sources into SASS district- implements these complex student level surveys as noted above. Second,one assignment plans using choice on a has to assume that basic commonalitiescan district-wide basis. Many of the districts be extracted from the large number of used their program funds to hire new "state" systematic studies currently being teachers and staff development for the conducted, even though they involve only magnet schools (besides substantial a few high-profile states.' Third, one investments at the district level in program must believe that the methodological split specialists).Implementation of new choice between qualitative ethnographiccase programs also required large-scale studies and larger scale semi-structured outreach programs to attract new students surveys will diminish.Fourth, there is an and the development of more extensive expectation that additional investigations of transportation plans to handle district-wide state-level differences using the 1993 choice. An interesting aspect of these SASS can replicate certain findings from choice mechanisms is the manner in which these reform studies.Finally, one has to districts handled admissions, the priorities assume that more comprehensive surveys granted in-the admission process, and the on the effect of reform implementation maintenance of waiting lists.Obviously if will be administered at the state level to there is high demand, individual compare different types of policies. At a applications have to be administered and minimum, a more realistic understanding centralized at the district office, even in of the difficulties should emerge froma medium-sized districts. The questionnaire critical comparison of different methods, offers some insight into the administrative even in an area with clearly established processes that are amenable to descriptive standards such as mathematics. "check-listing" as approximate summaries of administrative decision making. Other A valuable introduction to some of these reforms based upon choice mechanisms issues can be garnered from a recent (between-district plans, charter schools, Michigan State report (Spillane et al., and voucher proposals) need similar 199537). The methodological framework administrative structures to attract students for this study has developed froma series from large geographic areas. of evaluations (see Spillane, 1996, March) where the "key role" of LEAs in District-Level Studies of School Reform instructional policy making is apparent.' Michigan's reforms are representative of In this section, the potential for linking initiatives that are designed to radically SASS district-level data to the assessment restructure instructional practices in a state of state-level reform efforts is examined. and incorporate a specific set of state- Obviously, a variety of optimistic and developed policy recommendations as speculative assumptions permeate this outlined in "Essential Goals and evaluation of SASS's potential as a Objectives"39 that are linked to national
144 151 frameworks in mathematics (NCTM) and local perspectives on state and federal science (AAAS). The objective of the policies. Michigan State study was to determine what local school districts were doing to At this point, it is necessary to review the reform mathematics and science education broad outline of national NCTM standards (i.e., what changes were occurring) and and some of their implications for state what "influences the way local schools policies before district implementation can make policy about mathematics and be discussed (this is classic top-down science education." reform). At a general level, these standards outline a general set of topics Nine districts were selected based upon organized around four basic themes geographic location, district size and urban (problem solving, communication, type, social and ethnic composition of reasoning, and connections) and then more student population, and "reputation" for grade-specific recommendations for reform activity.It should be noted that alignment of content coverage.' In this the range of variation rather than statistical study, the state document Essential representation was the primary Principles was generally recognized as a consideration in district selection. set of new policies that required substantial Accordingly, the study included two large changes in curricula, instructional mid-size city districts with high-minority practices, textbooks, and so forth, by all population and high percentages of "free districts. The conclusion of Spillane et lunch" students, but it also included al., however, is clear: "The reform smaller low-minority rural districts with rhetoric masks significant variability across substantial percentage of students receiving and within districts" (p. 34). While all "free lunch."' The interview selection districts indicated that they were process within each district is more implementing the new state guidelines, the complicated.First, central office details of specific reforms revealed distinct personnel with instructional responsibility differences in the priorities given to were interviewed; second, for the two different themes. Only three districts had elementary, one middle, and one high moved beyond more routine topic school from each district in the study, the identification toward substantive alignment principal was interviewed; and third, as it relates primarily to two criteria teachers with the "lead role" in (communication and reasoning) when mathematics and science education were compared to the other districts that gave also part of the study.' The total more attention to other themes (problem number of interviews ranged from 13 to solving and hands-on mathematics). More 32 per district, and these open-ended specifically, in these latter districts new interviews were then transcribed.' Six concepts became new "labels" for old categories were used to code the first activities ("hands-on" became the same round of interviews: background use of concrete materials information on the district, substantive "manipulatives" and "integration of ideas about mathematics and science, the concepts" became more group activities). efficacy of LEA policy, the opportunity The rich discussion in this report suggests for teachers to learn about policies, and an underlying rank order in the
145
152 implementation of these reforms thatare Without commenting on the specific measurable when one asks about certain findings at the school level, they found the topics for certain grades. "LEA actively engaged in instructional policymaking, both defining policy The findings for science demonstrate how problems and crafting solutions to them" seemingly parallel state frameworks (Spillane et al., 1995). This conclusion generate qualitatively different levels of contrasts with conventional perspectives on change. The AAAS science standards" local school districts as the passive emphasized connections and common "implementor" of state and federal policies themes between disciplines, teaching a (or more narrowly concerned with smaller number of central scientific ideas, administrative and budgetary issues ) and developing students' ability to utilize rather than directly concerned with scientific methods and technology. While instructional content.This active role in all districts were either purchasing instructional policy appears to bea new curriculum guides from outside sources or development in these districts, and the developing their own, these materials were variability between districts is more only aligned topically to the state striking when specific organizational and frameworks. In the four districts that had historical factors are examined.It is moved closer to state standards, the important to note that formal or "boundaries" between conventional (traditional) "channels" of influence had disciplines had been "softened," but there definite limits.Neither curriculum guides, also was more explicit attention to curricular materials, student assessment, principles of "constructivist learning" and nor professional developments were "conceptual understanding." Description initially influential in shaping mathematics of these topics, as district administratorsor and science reforms when traditional lead teachers explained how material was methods were employed. In a narrow presented differently, suggested an ability sense, most LEAs emphasized the simple to actively translate these principles into coherence of topics and utilized lists the curriculum.In many of the other ("what teachers should teach") rather than districts, reform was limited to "hands-on the more radical restructuring of ideas science" and these reforms, unfortunately, about "substantive reform ideas." There appeared to be quite similar to the old is no question in the Michigan State "cookbook laboratory experiments." analysis that two state laws (a mandated Likewise, integration of content from core curriculum and fiscal penalties for different disciplines (particularly poor performance) stimulated district mathematics) often evolved into team administrators to "pay attention to teaching, without adoption of a newer instructional issues for the first time." integrated curriculum. Despite this opportunity to use these mandates to leverage support fornew The Michigan State study then examined reform agendas within each district, the the more complex process of how these distinct variation between districts in their proscribed changes in classroom responses, as summarized above, does not instruction successfully flowed downward lend itself to simple a priori from these new district policies.' explanations 46
146
153 The conceptual approach proposed by participants to bring back to their districts Spillane accounts for these differences and a "sense of ownership of the reform relies upon district organizational agenda" which they then could capacities, organized access to communicate with more substantive information, and the skill of individual conviction to other teachers. The third administrators (such as knowledge and elements (time, funding, and labor) were commitment).47 A common theme across resources that district administrators these resource capacities was the continued to control before and after the mobilization of individuals within a district new reform initiatives.Small district size into a more focused and organized and state regulations often imposed serious collectivity.("This interaction of constraints on the ability of districts to organizational and individual resources is a shuffle priorities in order to give more key to understanding an LEA's capacity attention to new instructional issues. For for instructional reform.Spillane et al., example, staffing curriculum development 1995). Before the statewide reform was committees and funding substitutes while initiated, most districts had limited regular teachers spent one week in structural capacities to initiate new professional development seminars instructional initiatives.Administrative required administrative skill (and not structures were hierarchical and budgetary flexibility and available funds). preoccupied with managerial and procedural concerns (i.e., simple These case studies provide strong mechanical compliance)." The evidence, in my opinion, that the local development of resource capacity school districts will play a critical role in described in the Michigan State study have educational reform.State legislation has certain parallels to James Coleman's affected the broad parameters of reform in (1990) analysis of organizational terms of proposing core curriculum, new innovation and the creation of social statewide testing requirements, and funding capital in output-driven systems.In order incentives. But the state mandates cannot to construct new curriculum material, be directly translated into new instructional district administrators first had to identify practices at the school level without a knowledgeable experts within the district restructuring of the relationship between (usually "lead" teachers) in each schools and district curriculum policies as discipline, and then organize some demonstrated by these experiences in sufficient numbers of these individuals Michigan. At the same time, other reform willing to collaborate in a "new" strategies that have focused on individual enterprise.49 school-level reforms (where the district is bypassed entirely) have not been a Second, links to external professional "stimulus to change in individual schools networks' provided access to discipline- over time" (Elmore, Abelman, & specific knowledge for certain Fuhrman, 1995). In the context of the administrators and teachers. Besides Spillane arguments, individual schools do providing opportunities to learn about not have the administrative capacity or these new reforms first hand from these resources to mobilize curriculum reforms professional organizations, they allowed mandated by new standards (although a
147 154 limited number of high performance some "feedback" from principals, schools may have successfully teachers, parents (and probably their implemented them prior to the school board and superintendent) establishment of these reforms51). In concerning the "relative progress of Michigan, a permissive charter law will change."' Michigan is not unique in this provide interesting comparisons of "movement" toward "standards based" different strategies, even though these reform .56 The organizational structures charters must be "sponsored" by school of local districts have responded, maybe districts. more out of necessity than principle, to these pressures for improved student The final question, of course, is the ability performance from the state.District to consistently "track" different reforms in administrators have forged new different states.' At least for connections with school staff and teachers mathematics and science, comparative to design a more coherent, but not always "evaluations" have been started as part of consistent, set of curriculum principles and the NSF state systemic program (SRI, instructional guidelines. In most cases, the 1996)," and individual states receiving districts have not passively removed SSI grants have also conducted their own themselves from the process and they have studies.' There may be sufficient not allowed individual schools to mobilize information from all these studies to existing capacities or develop new extract some common themes concerning resources for these new standards. the role of district administration in developing instructional content in their schools. Some precipitating event (usually Conclusion new state requirements, tests, or curriculum policies) provides an The current and future utility of SASS is identifiable context (when did this occur, derivative of these relatively new state what did the district think it would have to education reforms. do, etc.). Then the process of implementation involves several key The only comparative state data on the elements: How was the new curriculum organizational capacities of districts and content constructed? Who was involved, schools comes from SASS. Short-term how long did it take, what financial student outcomes from state-level NAEP resources were shifted? What was the are important, but between-state variation impact of new testing standards? Were often are not as critical as between-district new forms of professional development comparison within a state to state-level organized? The articulation of different policymakers. They are more concerned themes in the new mathematics standards with their own performance systems and suggests that how administrators "talk" the quality of instructional capacities and about reform has some relation to what resources within their own state (more they have done to develop new material, importantly how they have changed over and how they have gone about time). Accountability within existing implementing "reforms" in their schools. governance structures is an active force And, these administrators usually have driving these reforms.State education
148 155 commissioners are now more attuned to "choice" mechanisms in student governors and state legislatures, and assignment (at a certain level, this is how school superintendents are more responsive public schools respond to market pressures to their local constituencies. for private schools, charters, and Administrative and school organizational vouchers). The slow demise of the processes have changed accordingly, and neighborhood school" presents new this change has implications for how a problems of matching parental preferences survey is organized and designed. to more distinct educational offerings.If within-school reforms did not work, First, district administrators are no longer districts had to shift to district-wide exclusively concerned with routine magnet schools and more limited choice budgetary matters. More important, the options.In the process, the allocation of details (the dollars, the personnel counts, instructional staff is also subject to similar the number of students, closing old pressures. Total-school magnet principals buildings, etc.) are now collected and are usually allowed to choose most of their reported on an accurate and regular basis staff from any school in the district when in the CCD surveys. When the National the school first opens. What district Center for Education Statistics comes back administrators know (and what school for the same information from the same principals do not) is the complexity of administrators, the "bureaucratic response" shifting students, staff, and finally federal may vary. The numbers may have and state money associated with programs changed in the intervening months, certain for special populations between schools. types of details may never have been In most cases, these processes are not available, or the terminology may not be random and cannot be reconstructed recognizable to administrators in certain without understanding the process. types of districts.Large districts are fundamentally different from small At a minimum, district administrators can districts.Accordingly, questions that describe what they have done, or at least overlap with CCD surveys should, in what they have been doing, since some principle, be avoided or subject to a "new" policies were implemented at some simple and quick review (at the end of the fixed point in the recent past." The survey: just ask, "by the way, can you outline of these policies for student quickly confirm these numbers."') assignments has already been discussed. But the reform process also has to involve If one asks district administrators questions teachers and principals.Ironically, many that are more relevant to their day-to-day teachers have strong ties and professional concerns and problems (and which have investments in the school where they are not been asked before), they can give more "employed," but they are employed by the detailed, consistent, and informed district rather than the school.Subject to responses. The questions themselves have a multitude of procedural constraints, to be logically simplified with more teachers can be and are reassigned with introductory explanations to establish only limited "choice" by district common definitions. A major concern of administrators.In the 1980s, arbitrarily districts is the increasing preference for shifting personnel between schools was
149 156 common. The Michigan State study the policy dialogue as discussed by provides a different perspective on the Spillane et al.Maybe, if one continues to networks between teachers in different avoid the question, the significance of schools, administrators, and professional administrative structures will quietly organizations that are emerging in disappear. This probably will notoccur. response to new state content standards. States have chosen not to abolish local The outlines of these organizational school districts, and only under extreme capacities and the resources (often more conditions such as receivership have states time than money now) that are necessary decided to administer districts with state to mobilize before instructional change can personnel. The only alternative is to occur are still controlled by local school review, step-by-step, the implementation districts.Despite strong pressures for process of new reforms at the district level decentralization and deregulation, schools in each state. The administrativeprocess themselves have demonstrated limited reflects a common set of instructional capacity to initiate reforms except in themes and new accountability isolated cases. mechanisms. The major changes are compatible with a new and measurable The question itself about the future status discourse. of a LEA survey, in many ways, mirrors
150 157 Appendix I
STATE DISTRICT SAMPLING SCHOOL SAMPLING CCD Dist Dist School Ave. Dist School Ave. Ave > 10k Alabama 42 203 4.8 103 221 2.1 10.0 6 Alaska 15 136 9.1 46 196 4.3 8.7 3 Arizona 50 246 4.9 95 170 1.8 5.0 14 Arkansas 61 173 2.8 126 164 1.3 3.4 2 California 99 394 4.0 268 416 1.6 7.2 53 Colorado 31 301 9.7 74 173 2.3 7.6 16 Delaware 22 51 2.3 19 72 3.8 8.0 Dist of Col 1 35 35.0 1 72 72.0 181.0 Florida 20 400 20.0 55 258 4.7 37.0 19 Georgia 44 286 6.5 97 179 1.8 9.4 17 Hawaii 1 73 73.0 1 94 94 238 1 Idaho 36 167 4.6 79 166 2.1 2.3 4 Illinois 94 344 3.7 193 283 1.5 4.2 9 Indiana 61 218 3.6 132 184 1.4 6.2 10 Iowa 72 204 2.8 128 169 1.3 3.6 6 Kansas 57 243 4.3 110 161 1.5 4.9 6 Kentucky 42 201 4.8 98 167 1.7 7.9 3 Louisiana 29 297 10.2 67 225 3.4 20.5 15 Maine 54 173 3.2 105 152 1.4 3.2 0 Maryland 14 293 20.9 23 171 7.4 52.6 12 Massachusetts 67 212 3.2 157 229 1.5 5.0 7 Michigan 85 329 3.9 189 227 1.2 5.2 16 Minnesota 72 200 2.8 134 171 1.3 3.7 13 Missouri 76 249 3.3 126 178 1.4 4.1 13 Montana 70 137 2.0 155 161 1.0 1.8 1 Nebraska 71 204 2.9 116 170 1.5 2.0 3 Nevada 11 193 17.5 18 119 6.6 21.3 2 New Hamphsire 46 135 2.9 76 120 1.6 2.7 2 New Jersey 93 223 2.4 151 194 1.3 3.9 9 New Mexico 25 188 7.5 62 142 2.3 7.4 7 New York 63 384 6.1 201 313 1.6 5.3 7 North Carolina 41 325 7.9 92 184 2.0 14.9 21 North Dakota 47 136 2.9 130 162 1.2 2.3 2 Ohio 93 239 2.6 155 196 1.3 6.1 10 Oklahoma 72 230 3.2 235 161 .7 3.2 10 Oregon 53 255 4.8 107 170 1.6 4.1 7 Pennsylvania 88 225 2.6 159 196 1.2 6.0 6 Rhode Island 23 158 6.9 35 106 3.0 8.5 2 South Carolina 38 257 6.8 70 164 2.3 11.6 13 South Dakota 48 131 2.7 112 164 1.5 3.5 2 Tennessee 35 258 7.4 86 189 2.2 10.9 12 Texas 84 442 5.3 291 413 1.4 5.9 45 Utah 20 257 12.9 31 170 5.5 17.8 11 Vermont 62 91 1.5 92 108 1.2 1.3 0 Virginia 35 275 7.9 92 188 2.0 13.4 15 Washington 56 298 5.3 117 197 1.7 6.8 18 West Virginia 28 216 7.7 55 178 3.2 16.6 6 Wisconsin 64 219 3.4 126 170 1.3 4.8 7 Wyomin" 55 96 1.7 50 131 2.6 0 Total 2573 11379 4.4 6459 9333 1.7 4.4
151 OrEIEE191' AIDAtk ifa 1. 0 0 EDZQ0 iuvr mvAlluilmr References Chambers, J.(1995).Public school teacher cost differences across the United Baker, D. (1996). Towards an States (NCES 95-758). Washington, DC: organizational database on America's U.S. Department of Education, National schools: A proposal for the future of Center for Education Statistics. SASS. The Schools and Staffing Survey: Recommendations for the future (NCES Chambers, J.(1996). The patterns of 97-587). Washington, DC: U.S. teacher compensation (NCES 95-829). Department of Education, National Center Washington, DC: U.S. Department of for Education Statistics. Education, National Center for Education Statistics. Baker, D., Han, M., & Keil, C. (1996, forthcoming). How different, how Choy, S., Medrich, E., Henke, R., & similar? Comparing key organizational Bobbitt, S.(1992). Schools and staffing qualities of American public and private in the United States: A statistical profile, secondary schools.Statistical analysis 1987-88 (NCES 92-120). Washington, report (NCES 96-322). Washington, DC: DC: U.S. Department of Education, U.S. Department of Education, National National Center for Education Statistics. Center for Education Statistics. Choy, S., Henke, R., Alt, M., Medrich, Boe, E., & Gifford, D. (1992). Teacher E., & Bobbitt, S.(1993). Schools and supply, demand and quality. Washington, staffing in the United States: A statistical DC: National Academy Press. profile, 1990-91 (NCES 93-146). Washington, DC: U.S. Department of Bobbitt, S., Faupel, E., & Burns, S. Education, National Center for Education (1991).Characteristics of stayers, Statistics. movers, and leavers: Results from the Teacher Followup Survey 1988-89 (NCES Choy, S.(1996). Schools and staffing in 91-128). Washington, DC: U.S. the United States: A statistical profile, Department of Education, National Center 1993-94 (NCES 96-124). Washington, for Education Statistics. DC: U.S. Department of Education, National Center for Education Statistics. Bobbitt, S., Leich, M., Whitener, S., & Lynch, H. (1994). Characteristics of Cohen, D., & Spillane, J.(1992). Policy stayers, movers, and leavers:Results and practice: The relations between from the Teacher Followup Survey 1991- governance and instruction. Review of 92 (NCES 94-337). Washington, DC: Research in Education, American U.S. Department of Education, National Educational Research Association. Center for Education Statistics. Cohen, D. (1995). What is the system in systemic reform? Educational Researcher, 24(9), 11-17.
152
159 Coleman, J.(1990). Foundations of Levine, R., & Christenson, B. social theory. Cambridge, MA: Harvard (forthcoming). Public school districts in University Press. the United States:Statistical profile, 1987-88 to 1993-94. Washington, DC: Elmore, R., Abelman, C., & Fuhrman, S. U.S. Department of Education, National (1995). The new accountability in state Center for Education Statistics. education reform: From process to performance. Paper presented at Meyer, J., Scott, W. R., & Strang D. conference on Performance-Based (1987).Centralization, fragmentation, and Approaches to School Reform. school district complexity. Administrative Washington, DC: Brookings Institution. Science Quarterly, 32, 186-201.
Ingersoll, R.(1995). An agenda for Rollefson, M., & Broughman, S.(1995). research on teachers and schools: Teacher supply in the United States: Revisiting NCES' Schools and Staffing Sources of newly hired teachers in public Survey (NCES 95-18). Washington, DC: and private schools, 1988-1991 (NCES 95- U.S. Department of Education, National 348). Washington, DC: U.S. Department Center for Education Statistics. of Education, National Center for Education Statistics. Ingersoll, R.(1996). Teacher supply, teacher qualifications, and teacher Scott, W., & Meyer, J.(1987). turnover: 1990-91 (NCES 93-146). Environmental linkages and organizational Washington, DC: U.S. Department of complexity: Public and private schools. Education, National Center for Education In T. James and H. Levin (Eds.), Statistics. Comparing public and private schools. Vol. I:Institutions and organizations. Kaufman, S., Abramson, R., Cole, C., Philadelphia, PA: Falmer Press. Jackson, B., & Parmer, R.(1996).1993- 94 Schools and Staffing Survey: Sample Spillane, J.(1996, March). School design and estimation (NCES 96-089). districts matter: Local educational Washington, DC: U.S. Department of authorities and state instructional policy Education, National Center for Education Educational Policy, 10(1), pp. 63-87. Statistics. Spillane, J., Thompson, C., Lubienski, Leighton, M., & Mullens, J.(1995). C., & Reimann, C.(1995). The local Measuring curriculum content: The status government policy system affecting of recent work (NCES 95-11). mathematics and science education in Washington, DC: U.S. Department of Michigan: Lessons From nine school Education, National Center for Education districts.East Lansing, MI: Michigan Statistics. State University.
153 160 SRI International.(1996). Evaluation of the National Science Foundation's Statewide Systemic Initiatives (SSI) program: Second-year report. Menlo Park, CA: Author.
Stodolsky, S.(1996). Should SASS measure instructional processes and teacher effectiveness. The Schools and Staffing Survey: Recommendations for the future (NCES 97-587). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
Thompson, C., Spillane, J., & Cohen, D. (1994). The state policy system affecting science and mathematics education. East Lansing, MI: Michigan State University.
Twentieth Century Fund. (1992). Facing the challenge: School governance. New York: The Twentieth Century Fund Press.
154 181 1. The formal governance of public schools itself is another topic"... there is little agreement as to what the system actually looks like." (The Twentieth Century Fund, 1992)
2. For example, Baker (1996) and Stodolsky (1996) implicitly link individual teachers to classroom instruction practices through a greater attention to content-specific disciplines.
3. This paper unfortunately can not address any of the important organizational parallels or differences between sectors (Baker, Han, & Keil, forthcoming).
4. How this centralized and integrated political culture survived reform-oriented competition in the 1980's is another question.
5. This argument follows the discussion presented by Scott and Meyer (1987).
6. The percentage of funding from states sources (45.6 percent) now exceeds local sources (44.7 percent),State Comparisons of Education Statistics: 1969-70 to 1993-94.
7. The historical sources of this situation are traditional localism and federalism, mistrust of government, and political design.
8. The original argument was presented by Meyer, Scott, and Strang (1987), but has not been updated. Increasing sophistication in aggregate district-level and school-level data in CCD (supplemented by individual data from SASS school and teacher surveys) provide untapped resources to verify changes in these theories of bureaucratic complexity.
9. Carnegie Forum on Education and the Economy,A Nation Prepared: Teachers for the 21st Century, the Report of the Task Force on Teaching as a Profession(1986). The most detailed review is E. Boe and D. Gilford,Teacher Supply, Demand and Quality(1992).
10. The most comprehensive review of SASS-related research is contained in Ingersol (1995).
11. The school-level question asked only if there were any unfilled position, not how many.It should not be expected therefore that aggregate school-level numbers will provide precise estimates.
12. The limitations of the LEA survey, which can only ascertain how many teachers were new hires in that district, are quite apparent compared to individual data. A substitute measure based on net aggregate changefrom CCD would probably be satisfactory.
13. The sophisticated weighting of schools and districts ensures close estimates. Nonetheless, there are some inconsistencies in district-level responses such as counting K-12 but including pre-kindergarten counts in the CCD. The time frame sometimes is important with districts giving numbers as of the date when they are filling out the questionnaire rather the date specified in the SASS survey form.
14. For example, the 1993-94 SASS public school sampling design is based upon teacher counts from the 1991-92 CCD (see Kaufman et al., 1996).
15. When a LEA does not respond, simple variables, such as district size, for example, are treated as missing in SASS user files.
16. There are seven functional categories including special regional purposes. Approximately 1,197 districts (1992-93 CCD) have either no (or only one) school, and no students (but some also report FTE teachers). Schools in these districts are excluded from the sampling frame and therefore these LEAs properly are not included in the district frame. In the 1993-94, there was a process to sample these teachers, but only a small percentage were actually found to be teaching in regular districts.It is tempting to recommend a "footnote and exclude" philosophy for these districts with minimal staff and small numbers of students.
155 162 BEST COPY MADLABLE 17. Based upon CCD estimates, only 53.8 percent of the total public school FTEare teachers.
18. Individual decisions are often made by the principal or hiring committee from filtered listsprepared by central office staff.
19. With a five-year interval, the district and school survey should ask whether policychanges occurred during this period and if so in what year.
20. In many large districts, attendance zones can be complex (particularly when choice optionsare available to some students), and they are established (and often adjusted on a yearly basis) exclusivelyat the district level:
21. The number and scope of these programs is well beyond the scope of thispaper, but evaluation of these programs has frequently involved representative samples and structured surveys.
22. The discretionary factors that districts can control were primarily measured by teacher characteristics (undergraduate major, highest degree, type of certification, etc.) rather than district-level data suchas base salaries, etc.
23. It should be noted that "climate" variables constructed from aggregating individual teacherresponses to the school level had large impacts. While also accounting for some of the missing schools, aggregation isan alternative strategy to estimate higher-level units.
24. These equations allow one to estimate whether opposite race teachers (minorities teaching in white schools and whites teaching in minority schools) had higher salaries controlling for the racial composition of the district. The major effect was for Hispanic male teachers to have higher salaries than white male teachers, and this effectwas not dependent upon the percentage Hispanic in the school or the district.
25. In some preliminary estimates, the average number of schools per district for the state of Michigan wouldincrease from 1.2 per district to 4.8.
26. Florida has a total of 67 districts.
27. There are still over 6,000 districts with less than 600 students. Under this proposal fewer schools would be sampled and a brief (if any) LEA instrument could be administered, thereby reducing total burden.
28. In North Carolina, there are seven schools with student populations greater than 25,000 andan average of 19.7 schools (the range was 10- 37 schools) were sampled. Among the 14 smaller districts (population between 10 and 25 thousand), an average of 6.8 schools per district (the range was 3 to 13 schools)were sampled.
29. The size of the district when a desegregation plan was first implemented (nota district's size when its current plan is assessed) is important to consider in order to avoid confounding growth in student population dueto racial composition of the district and its metropolitan context with the long term effects of the plan independent of its specific components.
30. Complex student assignment plans utilizing choice are typically not employed in smaller districts.In their initial plans, all-black schools were closed and attendance boundaries or grade structures were changed.
31. There was an additional condition concerning special needs students.
32. For rough comparisons, the AIR study estimated that only 230 districts had magnets. Thiswas a telephone survey that included lengthy descriptions of what magnets were and their objectives for desegregationpurposes.
33. The published figure in 1993 Statistical Profile is 6.5 percent in reference toprograms offered within schools in a manner comparable to bilingual or Chapter One (Table 2.4).
156 163 34. The number of districts with this erroneous classification was only nine after the revised definitionof a magnet school was used and some may have been nonresponses, such as Chicago and St.Paul.
35. The "open enrollment" question did not provide meaningful numbers. In most smalldistricts (with one type of school for each grade level), all students were in "open enrollment" programs. Between schooltransfers is an unfamiliar concept.
36. The criteria for state selection are driven by standards-setting criteria, primarilyincreases in the number of mathematics and science credits required to graduate from high school. Comparable state data oncredits, revisions of guidelines to align with NCTM standards, and test requirements for graduation arecurrently reported in Table 17 in State Comparisons.
37. This ongoing study was funded by the Michigan Statewide Systemic Initiative underthe direction of James P. Spillane.
38. This an ongoing three-part study that is properly classified as a "policyimplementation" research.
39. The development of these new standards is contained in Thompson, Spillane,and Cohen, The State Policy System Affecting Science and Mathematics Education in Michigan (1994).
40. This type of district selection strategy, common in most state-levelevaluations, implicitly reflects the interest to find the widest range of different types when only a limited number ofdistricts can be studied in depth.
41. In principle, 36 schools would constitute the selected school samplefor these nine districts, although in the smaller districts fewer schools were part of the sample. The 1993-94 SASS sampled227 schools (7 percent) in Michigan, based upon the number of teachers in the state.In general, Michigan (558 districts) has a large number of smaller districts with 27.1 percent more of its students in districts less than 2,500compared to national levels.
in the Reform Up Close study. 42. More structured interview protocols for district-level personnel were developed There are certain characteristics of district-specific policy that appear tobe "common": is there a "framework" document that the district has adopted, who decides what textbookswill be used, is there a testing program in the district, and have there been changes in graduation requirements.
43. This approach differs from the more detailed taxonomy used in the ReformUp Close study (see M. Leighton and J. Mullens, Measuring Curriculum Content: The Status of RecentWork) designed to measure instructional content at the classroom level.
44. The National Research Council has proposed a different set of standards.
45. The actual classroom implementation of these reforms is thethird phase of this study. Of course, how comprehensive district-level curricula facilitates more fundamental changes ininstructional practices within schools (even when background variation in content knowledge is considered)provides a rigorous test for this district mediating theory.
46. From the perspective of developing a district-level survey in the contextof SASS, it is not clear that retrospective accounts at a single point in time can reconstruct how moresuccessful and articulate themes were developed by administrative personnel.
47. Six factors are identified in the study: knowledge, commitment anddisposition, time, funding and labor, professional networks, trust, and collaboration.
48. The obstacles presented by bureaucratic layering (particularly in largedistricts) have to be considered a factor in the slow implementation of reform.It seems that districts had the necessary resource capacities, but had to make specific decisions to mobilize content knowledge and utilize this expertisein a different manner. Different allocations of personnel, time, and funds were critical in the "crafting" ofdistrict-wide policy.
157
164 f3EST COPY AVAILA:LE 49. From the descriptions in the study, the necessary "critical mass" was a mixture of self-selection(teachers volunteering themselves to lead the reform) and decisions by district administrators about theirlevel of expertise. Apparently, conventional selection standards (no rulesseem to have guided the selection) for committee participation were not followed and a decision was made to include teachers from most or all schoolsto bolster the argument for "representativeness."
50. These include NCTM, Michigan Council of Teachersof Mathematics, Michigan Partnership forNew Education's Frameworks Project, Michigan-funded Mathematicsand Science Centers. A substantial amount of federal Eisenhower funds flowed to the organizations through professional development programs (allocatedto the districts).
51. The issues in school-level reform, primarily deregulation to promote autonomy and innovation, oftenfind themselves limited to "relatively successful schools, to which eligibility was generally limited, did not findmuch need to embark on wholesale change, and used deregulationas one of many resources to support innovation." (Elmore, Abelman, & Fuhrman, 1995).
52. Unfortunately, the more detailed case studies have focusedon implementation in individual states and themore comprehensive, comparative state analyses focus on national standards without providing analytic methodsto examine between district variation within states.
53. CPRE's new Center, funded through the GovernanceInstitute, has proposed additional analysis ofstate and local reform policy in these 26 states.
54. It is extremely doubtful that many of the state-level studiesare comparable in insight and depth to the Michigan State study.
55. The Michigan State study included some interesting materialon "complaints" (i.e., why the state policies were "unreasonable," why they didn't have moneyor personnel to prepare the mandated guides, etc.) that help distinguish the relative progress of different districts.
56. In a survey of 50 state school superintendentsor commissioners, 43 claimed that they were revising their assessment and accountability systems in accordance with these principles(Elmore, Abelman, & Fuhrman, 1995).
57. Most of the first 22 questions are eliminated by this criteria.
58. Fixed geographical attendance boundarieswere more amenable to sophisticated geocoding schemas and computerized transportation programs. These administratorsdidn't have to visit a school or talk toa parent. 59. Despite the idiosyncratic labels embedded in district-wide reforms,there are common features below the surface.
158 165 USE OF EDUCATION INFORMATION SYSTEMS WITH THE SCHOOLS AND STAFFING SURVEY: HOW CAN SASS BE LINKED TO SCHOOLS?
Rolf K. Blank, Council of Chief State School Officers
The Schools and Staffing Survey (SASS) monitoring and reporting on the status of has proven to be a very important and education systems to Congress, federal useful multi-purpose survey of teachers, agencies, and states; and a valuable source administrators, and school systems. The of data on teachers and administrators that next iteration of the SASS now being is useful for analyzing current conditions planned for 1998-99 school year can build in K-12 education and projecting needs for upon a record of success and educators. accomplishment from the 1988, 1991, and 1994 surveys. SASS is accomplishing My main position in this paper is that the some important goals; in the 1999 survey current design of SASS could be it can reach for a new goal of relating significantly strengthened through linking survey data on teachers to more detailed with education information systems and data from school information systems on that it would provide important data on the the students teachers teach and the schools characteristics of American schools and where they work. how education is carried out within them. My suggestions and proposals for use of SASS results provide important national- data from education information systems and state-level statistics for American are based on my assessment of the educators about critical areas such as strengths and weaknesses of the current characteristics of the teacher and survey and my views of the needs of administrator force in elementary and education decision makers and educators secondary schools, basic descriptive for better information about schools and statistics on dimensions of schools, and staff. reliable data on the preparation and experience of teachers in their assigned fields and positions. SASS provides State and Local Education critical national statistics on projections of Information Systems Versus SASS teacher supply and demand and key variables for analyzing trends in teacher Strengths of SASS Design turnover and hiring. SASS is based on surveys of individuals I approach this paper for the National from stratified national and state samples Center on Education Statistics (NCES) of schools. The strength of the design is from the perspective of educational uses of in providing reliable national estimates SASS and the data produced from the from samples of teachers and surveys. SASS meets several important administrators with sufficient numbers of purposes: reliable, periodic statistics for
159 respondents and items to provide both of SASS as a general-purpose education detailed descriptive information and survey are: relational analyses using a large number of variables. Another strength of the design Information about the students in is providing reliable state-level estimates schools across a wide range of variables describing Lack of curriculum and instruction teachers, administrators, and schools. In data some areas of statistics about education, Student outcomes SASS provides breadth and depth of How schools are organized and information not available inany other how resources are used national or state survey, including: Capacity to analyze equity of staff and resources allocation in relation Teacher and administrator to student characteristics background Teacher education and experience SASS has further limitations at thestate Current assignments level. Due to limitations of sample size in Pay and benefits of educators each state (100-200 schools), important National projections of supply and uses of SASS data at the national level are demand not possible at the state level, suchas Student enrollments and staff analyzing extent of teacher turnover by characteristics analyzed by key field, projections of teacher demographics--size, type of supply/demand, and trends in preparation community, state of teachers by field.
SASS is designed from a research-based Design of Education Information Systems model of key statistics for understanding and explaining the operation of the State, local, and school education teaching force in schools. Using a sample information systems are typically basedon survey, SASS can ask more questions from universe data collection. These data a sample of teachers and schools than systems have been designed mainly from would be possible or feasible ina universe the requirements of state legislation and or population data collection. A sample district policy--that is, the systemsare survey across states also provides a designed to provide counts and tabulations standardized method of collecting dataon on students, schools, and educators in basic elements of the education system, order to satisfy laws and policies such as number of schools, students, concerning education accountability, special programs within schools.States monitoring, and reporting. An example collect data on these variables but theyare would be reporting the extent to which hard to standardize. schools have state-certified teachers,or assessing the extent of improvement in Limitations of SASS Design rates of graduation.
The SASS design emphasizes certain Recently, more states and districtsare purposes and uses. The main limitations designing data systems basedon rational
160
167 planning of the multiple purposes and uses There are three possible approaches to of data at all levels, and the efforts include linking SASS with data from education systems for linking and transferring from information systems. All of these classroom to school level to district to possibilities reflect the goal of increasing state. However, even the most recently data analysis and reporting at the state-by- developed "unit record systems" such as in state level. SASS has made strides in Texas, Florida, and Ohio have not providing state-by-state data on education attempted to expand the breadth of the data (NCES, SASS by State, 1994). My view into many of the areas covered by SASS, is that SASS can significantly strengthen such as teacher background and education, the useability and relevance of the Survey professional development, attitudes, or to educators by linking to education conditions in schools (CCSSO information systems that will greatly [unpublished analysis of state augment the capacity for reporting at the accountability and indicator reports], state level and developing within-state 1996). analyses.
All states have an education information State Aggregate Statistics system that involves collecting data from all districts and schools. About 45 states SASS results are used to produce maintain annual databases for all teachers indicators at the state level. The National and administrators with their Education Goals Panel, Council of Chief assignments/courses and other information. State School Officers, and the National About ten state systems include student Science Foundation are among users of records with student background data and state-level statistics, mainly data about annual data on variables such as student teachers, which are generated from SASS. activities, courses, and test scores. It would be possible to generate state totals and averages for indicators not available in SASS, such as: Why Use Education Information Systems? Course enrollments Enrollments in special programs Graduation rates Given the main differences between the Student and teacher transfers purposes and scope of data in SASS and between schools and districts state or local systems, what are major Detailed data on teachers by reasons for trying to link data from education information systems to the assignment SASS? What advantages and benefits are These state aggregate statistics could be there are for NCES, the sponsoring agency? What advantages and benefits are generated by state education information there for states and for local districts and systems using definitions and categories supplied by SASS, allowing a match to schools--that is, the expected partners in the project? What are the advantages and SASS statistics. For example, state data benefits for customers and users of SASS could be used to report the percentage of teachers in each state in the subjects/fields and education data?
161
168 defined on SASS questionnaire that are disadvantage of this approach isthat data over a given age or are male vs. female. would not be available for analysisdown to the district and school level. In 1991, NCES conducteda field trial with ten states by asking them to report state- State aggregate data on science and collected data on specific SASS items, mathematics enrollments, teacher including teacher assignments, assignments and demographics, and certification, and special program teacher certification have been collectedby enrollments. A workshop held by CCSSO CCSSO from SEAs since 1990. to analyze the results showed that there Currently, 35 states reporton all of the were differences in how states defined and state aggregate statistics.Forty-five states reported some of the variables in report on teacher assignments for grades 7- comparison to SASS and in comparisonto 8, 9-12 (Blank & Gruebel, 1995). each other.State data managers didsee the possibility of reporting data requested CCSSO also collects state policy by NCES at the state level thatare not information from all statesevery two reported by SASS, or variables that donot years. The policy information on have sufficient detail at the state level. graduation requirements, standards, State managers saw strong benefits for teacher certification, attendance, decreasing burden on schools of reporting accreditation, and policies on timecan be available data from informationsystems to made available to SASS (CCSSO, 1995). avoid duplication or further need for expansion of SASS. States Report Data at State and School Level How could this approach (#1) be carried out? NCES would need to contract with each SASS selects a sample of schools for the state, or an organization that would nation and for each state.Teachers and subcontract with each state, to produce administrators are surveyed from the data according to specifications and sample of schools. At the school level, definitions defined by SASS. The amount the SASS survey collects dataon student of money would not be large for each and teacher characteristics suchas state, but a contract would provide for a race/ethnicity and on enrollments in timetable, staff to perform the work, programs such as Title I and vocational reporting schedule, a state commitment, education. In 1994, SASS requested data and assurance from NCES aboutuses and on a sample of students from each school reporting of the data. The advantage of in order to determine school-to-school using state aggregate statistics reported by differences among students. states is that SASS could obtain additional state-level indicators in key areas of policy SASS is limited in reportingon and and educator interest, such asprogress of analyzing differences among states in secondary students to higher levelcourses, school-to-school variation in areas suchas with relatively little additional cost and student background, teacher with the addition of 50 data collection sites characteristics, and schoolresources and (state departments of education). The materials. Four important questions could
162 be answered by linking SASS with state (C) What are differences in curriculum information systems for a sample of offerings, programs, and course schools in each state. enrollments according to differences among schools and (A) Is there equity of access, by state, students? to well-prepared teachers and resources for excellent teaching for Many states, not all, would be able to students from schools with varying report school-level data on courses and composition and location? programs. Other states would have to request the data from districts or schools. These data would provide the kind of These data would not provide data from research analysis that Jeannie Oakes, Iris teachers on subject content and instruction Weiss, and others have conducted with the (see below) but they would be able to National Survey of Science and provide universe data for all students in a Mathematics Education (see, for example, school to determine differences by school Weiss, 1994). The NSSME surveys characteristics matched with teachers in the (1977, 1987, and 1993) were conducted same schools. with a national sample of teachers and schools. The results of the analyses (D) What are the differences in showed significant variation in the expenditures by function (classroom allocation of well-prepared teachers to high teachers, administrators, benefits, minority vs. low minority and high-SES buildings, textbooks, materials, vs. low-SES schools. Also the issue of technology) across school course taking differences by school/student differences? characteristics could be studied. This kind of analysis can now be carried out with Recently, systems have been developed for SASS data at the national level. coding and analyzing education expenditures down to the school level. (B) What is the rate of teacher turnover These systems involve a methodology for and return to teaching, by state, coding all education costs and allocating across different subjects/fields? costs on a per school and per classroom basis. SASS produces important and Little can be done with teacher useful data on teacher and administrator supply/demand questions at the state level salaries and associated costs.If a state, using current SASS samples. By adding districts, and schools used the same universe data from all schools in SASS accounting system, SASS data could be sample, including teacher assignments, linked to data at school levels and costs demographics, number of years teaching, other than salaries could be analyzed to etc., NCES could join with states in determine how education funds are used analyzing turnover by field/grade level and across states and by school characteristics. by school characteristics as well as analyzing some data on teachers returning How could this approach (#2) be carried out? to teaching and new teachers. NCES could contract with states to report universe data on all students, teachers,
163
I 7 0 courses, programs, and costs for schools each school to assist with the data selected into the SASS sample. This is reporting/transfer. based on the assumption that the SASS school sample is representative of schools By drawing on existing school records and in the state.If some additional issues data systems, SASS could aska focused pertaining to school organization, set of questions that would get at the programs, and curriculum were desired, following key areas of data that would such as effectiveness of the middle school reflect key policies and school decisions, approach, additional schools would need to operations, student and teacher be selected. assignments, and student activities and progress. The kinds of data envisioned Data on the universe of teachers and with this approach are similar to the kinds students in the sample schools would allow of items in the NELS surveys. All of the for analysis of the questions about equity, variables identified in #2 could be teacher turnover, curriculum, and finance. collected with this approach. Additional NCES could establish a system of variables that could be accessed: technical assistance to states for sampling and for matching data definitions in SASS Changing student and teacher with definitions used by states, or for data composition: number of new collection and reporting. This model of students per grade per year, assistance to states is employed by Centers number of new teachers per year for Disease Control in its Youth Risk per subject/grade Behavior Survey that is conducted in collaboration with states. Allocation of materials, texts, and resources:e.g., data on the types School as Unit of Analysis--Working of books, equipment, and other Directly with Schools and Districts curriculum materials and how they are allocated Data from schools and districts. A third approach would be to expand the number Student outcomes: type of student of variables for school-level data collection assessments and other methods of through education information systems. evaluating progress, how The 1994 SASS tried a student record data assessment data are used (public collection using a sample of students in reports vs. teacher diagnostic and selected schools. This was designed to curriculum planning), aggregate provide statistics on the characteristics of school or class-level scores on students in each school. SASS could also tests, student retention and link to school-level data systems and promotion, suspensions and expand the kinds of questions to be asked transfers/reassignments at the school level.This model assumes that SASS could send a data diskette Curriculum information: data that directly to a school and they could transfer reflect curriculum and instruction in school-level data directly to the diskette. schools may be obtained from NCES may consider a small payment to information systems, including
164 171 course offerings, levels of courses Efforts should be made to collect and and enrollments, teachers assigned provide much broader access to data about to courses schools and what goes on in schools, at both national and state levels. Data on teacher inservice and professional development: schools or districts would be more likely to Incentives for Schools,IIistricts, have data on professional States development in an information system, possibly as part of their NCES and SASS needs to strongly personnel files. To the extent these consider an approach to data collection that data are annually updated, a wealth will provide some incentives for of data on current professional cooperation. NCES and federal data development activities may be collection has typically approached available education units as "data providers." Users are viewed as Congress, researchers, or How could this approach (#3) be carried out? policymakers. Educators know this. This school or district approach would add Surveys are viewed as a burden, a major data collection component to particularly those for which they seldom SASS. Several thousand schools in the see results or direct feedback. In order to SASS sample would receive an additional make use of Education Information request. By 1999, data systems and access Systems, NCES/SASS will need to to personal computers in districts and carefully think through and plan schools would be sufficiently developed to incentives. Here are suggestions: allow a computerized data collection form, saving time and effort for school ® Money: Small amounts go a long personnel. Data systems and coding way toward reducing negative structures may be common enough to responses allow a direct data entry/transfer approach that would expand the amount of Direct feedback: Give back school information that is currently collected from averages, state averages, or schools. The expansion of data collection national averages as soon as they from school level may not be possible if are available to all respondents. SASS relies on paper forms to be filled out Give them back a sample of the at the school level.But, even if this data results that are allowable method must be retained, a number of within the law. Make it a two- to additions could be made at the school five-page feedback of some kind level. Give educators something else in Schools are a key unit of analysis for return: NAEP has alleviated some research on education reform and analyses hard feelings by allowing schools to of education programs and curriculum, use keep calculators, science of resources and staff within schools, and equipment, and other assessment efforts to improve the skills of staff. materials. A report on the last
165
172 SASS survey could be givento Blank, R. K., & Gruebel, D. (1995). each school when the requestsare State indicators of science and mathematics made education 1995. Washington, DC: Council of Chief State School Officers. Raise the visibility and importance for educators: A letter froma Council of Chief State School Officers. congressman, senator or state (1995).State education policieson K-12 superintendent could be enclosed curriculum, student assessment, and saying how the survey and dataare teacher certification: 1995. Washington, important DC: Council of Chief State School Officers. Invite educators to a workshop in each region or state: NCES puts National Center for Education Statistics. lots of money into the Cooperative (1994). SASS by state, 1990-91Schools Statistics program with states. Use and Staffing Survey: Selectedstate results some of it to give and get feedback (NCES 94-343). Washington, DC: U.S. directly from educators in states Department of Education, Office of about data and their uses Educational Research and Improvement.
Think creatively about education Weiss, I. R. (1994). A profile of science data helping education: Dataare and mathematics education in the United often viewed as part of monitoring, States: 1993. Chapel Hill, NC: Horizon accountability--an external Research, Inc. requirement. How can it be viewed and provided as a support to educators? Efforts to build bigger and better databases will not be widely used or appreciated if this question is not addressed
References
Blank, R., Clements, B., Dalkilic, M., & Solomon, L. (1991). NCES/CCSSO workshop on improving reliability and comparability of staffing data. Washington, DC: Council of Chief State School Officers.
166
173 COLLECTING REPRESENTATIVE DATA ON SCHOOL RESOURCES: UNDERSTANDING THE LINKAGE BETWEEN ADEQUACY, EQUITY, AND OPPO i TUNITY TO LEARN THROUGH SASS
Jay G. Chambers, American Institutes for Research
Introduction and Background The paper begins with a definition of what is meant by the term school resources and The purpose of this paper is to suggest the importance of these data. This is ways in which the 1998-99 Schools and followed by a discussion of an alternative Staffing Survey (SASS) might be source of data on school resources: redesigned to examine patterns of resource namely, the fiscal reports of the Common allocation in public and private schools in Core of Data (CCD). The major portion the United States. The paper assumes that of this paper is devoted to ways of the reader has a basic knowledge of the redesigning SASS for the purpose of current structure and content of SASS. gathering better data on school resources. This section discusses ways of gathering This paper argues for taking advantage of data on the quantities of school and district the opportunity that SASS currently offers personnel, the qualities of school and to expand and improve upon the resource district personnel and compensation, data at the school and district level.It information on the state regulatory and suggests some significant reconfiguration funding environment within which schools of the ways in which personnel data are operate, utilizing the existing student gathered at the school and program level. questionnaire to gather data on service and It argues that SASS could serve as the cost profiles of students, the importance of foundation for considerable work of linking SASS to other databases, and, interest to other agencies within the U.S. finally, what we might give up in SASS to Education Department and could be make room for the proposed changes and utilized as a centerpiece of data analysis revisions. for studies of educational programs. Moreover, an enhanced version of SASS What Is Meant by School Resources? could reduce the need for more detailed fiscal data collection at the school or For the purpose of this paper, the word program level. SASS also could enhance resource can be used interchangeably with the student-level data collection to improve input.School resources are the ingredients the quality of information on how or inputs used in the educational processin individual students are being served, and the nation's public and private schools. SASS could begin to consider ways of Among these ingredients are the linking to other databases on student characteristics and capabilities of the outcomes to address productivity issues in students themselves as well as the school education. personnel and non-personnel resources used at the school and district level to organize and provide educational services.
167 174 Each student arrives at a school witha For all children to have equalopportunity unique set of endowments that when to learn, it is a necessary condition that combined with the resources provided by they have similar access toan appropriate the school produces a certain collection of level of resources. The nextstep in this outcomes, i.e., student performance, process is linking school resources to behaviors, and attitudes. student outcomes. Increasingour understanding of the linkage between Among the school resources are the resources to outcomes is going to be variety of personnel who come into direct critical if the push for higher academic contact with students. These personnelare standards as a cornerstone of school characterized by a set of attributes, reform coming out of the recent Education abilities, and skills.Their job titles Summit is to have any meaning. describe the ways by which they interact with students, while their personal The value of school resource datamay be characteristics provide information that found in the kinds of studies that couldbe may relate to their potential for generating done. outcomes for students. While there area number of other important non-personnel Resource allocation studies. What kinds of resources (e.g., computers, televisions, resources are being allocated to different and other technology items as wellas programs? What are the quantities and access to the Internet and its utilization) characteristics of the resources being which affect the process, thispaper will utilized in different communities? How focus attention on the personnel quantities are services being delivered in the nation's and characteristics. The fact thatmore schools? To what extent are different than 80 percent of public school district kinds of services being offered and how budgets are devoted to personnel costs are resources organized for the provision suggests the importance of understanding of services (e.g., through self-contained how this input is allocated and utilized classrooms, integrated or inclusionary among different kinds of schools. programs, pull out programs, departmentalized instruction)? What other Why Are Data on School Resources kinds of supplemental or related services Important? (e.g., physical therapy, occupational therapy, speech therapy, counseling, social By understanding the patterns of utilization services) are being offered? Whatare the and allocation of personnelresources, patterns of inequity in resource allocation? policymakers can increase their ability to What district characteristics contributeto address issues of equity, adequacy, and differences in resource levels? To what opportunity to learn. At the heart of the extent do community factors play a role in issue of adequacy are measures ofaccess resource allocation? How much can be of the nation's youth to educational affected by policy changes? resources, while the critical equity issue is the distribution of those resources among Estimates of programmatic costs. Whatare different types of communities andamong the patterns of variation in the allocations students with different educational needs. of resources among different educational
168
175 programs? How well do these programs and compensation of staff compare? What relate to the patterns of variation in student factors underlie the differences in the needs? What are the implications for the quantities and characteristics of personnel adequacy and equity of existing patterns of resources in public and private schools? resource allocation? Why SASS Versus the CCD- fiscal Reports? Educational productivity. What are the relationships between school inputs, For years, school finance researchers have student characteristics, and outcomes? paid considerable attention to the equity Which kinds of school resources make the with which educational resources and most difference? Are there differences services are distributed among local among types of communities in the schools and districts. The problem with effectiveness of school resources? To these studies is that they have focused on what extent can differences in outcomes be fiscal measures of resources which, accounted for by differences in school because of geographic or inflationary cost resources? differences, make it difficult to sort out the real differences in the levels of resources School finance reform. What impact do across schools and districts. NCES is different school finance formulas have on currently organized in such a way that the levels and allocation of educational creates a rather narrow view of what resources among different communities? constitutes school finance research. The To what extent do differences in district division of NCES within which the versus school characteristics influence the Common Core of Data resides is level of resources allocated to individual responsible for the school finance agenda schools? of the entire agency. The phrase "school finance" itself engenders an image of School labor markets. What are the dollar allocations. The problem is that differences in the costs of different school these dollar images provide little personnel? What factors affect the ability information about differences in the "real" of different local communities to recruit resources devoted to education. Spending and attract qualified personnel? What differences over time or across geographic kinds and combinations of school regions reflect both real differences in personnel are being utilized in different resources as well as differences in the communities across the nation? How do prices of comparable resources. such personal characteristics as degree level, field of study, type of certification, Another significant problem in fiscal data and professional experience match the is that it is ultimately organized according responsibilities to which they are assigned? to reporting standards that differ across states and over time. Although NCES Differences in public versus private schools. does publish an accounting handbook that What are the differences in the patterns of provides standards for reporting fiscal resource allocation and utilization in public data, not all states or local jurisdictions versus private schools? How do class use the handbook. Moreover, those sizes compare? How do the qualifications jurisdictions that do, do not necessarily
169 hold strictly to the accounting standards. Another issue confronted by NCES in In other words, there isa great deal of considering whether or not to begin variability in the standards of reporting gathering school level data is thecost of fiscal data among local jurisdictions. such a move. Doesone gather data on all Analysis of resource allocation must schools across the countryor a just a ultimately rely upon more detailed and sample of schools? To gather fiscaldata precise information on the physical on all schools requires imposition ofa ingredients utilized in theprocess of rigorous set of standards for reporting producing educational services. fiscal information. SASS providesa cheaper alternative by gathering dataon a In addition, there is much discussion from sample of schools, and it has the potential NCES as well as the community of school for providing raw data ina more finance researchers about moving to compatible format than iscommon in school-level fiscal analysis and about fiscal reporting systems. improving fiscal reporting to obtain estimates of how much schoolsare Many of the large-scale studies undertaken spending on different programs. SASS by the U.S. Department of Education may offer the best opportunity to obtain every year could benefit by a somewhat good data on costs of programs at the expanded version of the SASS. These school level through improvements in data large-scale studies require significant data collection on staff. SASS already collects collection activities that tosome degree a significant amount of important duplicate information that already exists information on school personnel. A within SASS. With some expansion of reorganization and reconfiguration of SASS and some consultation with those existing questions could enhance government program officers involved in significantly the value and the design of these large-scale studies, the comprehensiveness of the data (specifics costs of data collection for these other are offered below). SASS provides data studies might be reduced by virtue of the that are representative within statesas well availability of SASS. One role of NCES as across the nation. Representativeness of should be to provide the kinds of databases SASS within states permits comparisons of that permit analysts to conduct these kinds differences in patterns ofresource of large-scale studies of issues relatedto allocation that might be a result of the allocation of resources and to make differences in the administrative, these data available for use in the conduct regulatory, and fiscal environment within of studies being required by other divisions which schools operate across states.In of the U.S. Department of Education. addition to the added compatibility of Rather than having each studygo off in a information across states and local completely independent direction for data jurisdictions, using more directmeasures collection, the data for the SASS schools of ingredients in this way allows increased and districts and states could be usedas a flexibility to reorganize the datamore foundation for analysis. The advantage easily for different purposes. would be in efficiency in data collection, and it would provide larger samples of schools and districts for analysis formany
170 177 studies that currently rely on small samples files and with data on student of schools because of limited budgets. outcomes
For example, one could easily envision If it is necessary to sacrifice ways in which the SASS data collection elements of SASS in order to make could serve as a foundation for analysis of some of the revisions being resources utilized for special needs recommended in this paper, then it populations including Title I, special is suggested that SASS reduce or education, limited-English proficient, eliminate the subjective elements on gifted and talented, vocational, and early teacher attitudes and perceptions in education programs. With these issues in favor of more objective data mind, the following recommendations for revising SASS will be elaborated upon below. Redesigning SASS to Gather Data on School Resources Staff should be reported through FTEs rather than full-time and part- Quantities of School and District Personnel time head counts The current SASS school questionnaires Staff should be reported according request information on the numbers of all to types of delivery systems and full-time and part-time staff employed at programs they serve the school. No such data on district level staff are currently available in the SASS All staff should be covered in the battery of questionnaires. Two specific survey: both school-level staff in changes in the way school level data are the sample schools and district-level reported could improve the quality of the staff in the sample districts information for understanding issues related to resource allocation.First, More information on individual report personnel counts according to full- staff should be gathered to improve time-equivalents (i.e., FTEs) or in a way data on geographic cost differences that permits translation into the intensity of and inflationary trends--with ability services received by the school. Second, to control for personal personnel should be broken down by major characteristics program category. A sample of the way in which such personnel data might be Individual student information gathered is presented in the appendix to should be revised to obtain a more this report. The proposed changes are precise profile of needs and discussed in more detail below. services Reporting FTE personnel. As an alternative SASS data and samples should be to gathering the data on head counts of more effectively linked with other part-time and full-time staff positions, NCES data such as the CCD fiscal (e.g., as in items 16 and 17 of the 1993-94 Public School Questionnaire), these data
171 should be reported in terms of FTEs (full- might be defined as six hoursper day for time equivalencies) or hours per week of 181-day school year, whileone FTE services by job title of personnel. custodian could be defined as eight hours Certificated school personnel should be per day for a 210-day work year. reported in terms of FTEs because their jobs generally are defined according to Because of these potential complications in responsibilities for providing certain the way districts define FTE,one needs to categories of services and less in terms of gather information on non-certificated staff the time they spend providing them. Non- in a somewhat different fashion toensure certificated school personnel should be compatibility. For example, ifa school reported in terms of total hours per week employed two full-time custodians each of services rather than FTEs or head working 40 hours per week andone half- counts. time custodian working 20 hoursper week, the total reported in the cell in There are a number of ways in which this Table 1 for custodians would be 100 hours might be accomplished. For example, per week. If custodians typically worked a instead of just recording the number of 200-day school year, then the total of 200 full-time and part-time "school days would be reported in the column counselors," the questionnaire could labeled, "Typical contract daysper year of request the FTE number of counselors work for this category of worker. " employed at the school. This FTE count should reflect only the time of the Reporting staff by program. There is counselor devoted to a particular school. significant interest in the allocation of That is, a counselor who is a full-time resources among programs for special need employee but who works half-time at the populations. As shown in Table 1, the school for which the questionnaire is being school staffing data gathered on the SASS completed would be reported as a 0.5 FTE Public School Questionnaire could be counselor. An alternative approach would reconfigured to gather the FTE and hours be to maintain two columns in which the per week staffing data by program. number of full-time staff by job title and Rather than simply reporting totals for the total FTE or total hours per week of each category of personnel as in the part-time staff are recorded. current SASS questionnaires, reporting could be done for the major school Non-certificated staff are reported programs such as regular education, according to the total hours per week of special education, the Title Iprogram, service because there is a wide variation in programs for limited-English proficient the way districts define what constitutesa students, and early childhood education full-time non-certificated employee. In programs. Even if the decision is made to some cases, one FTE may be defined as an count the number of full-time and part- eight-hour day, five days per week, and 52 time staff, a matrix of job titles by weeks per year or a total of 2,080 hours of program categories could be reported to work per year, while in other cases one help assess the level of resources being FTE might be different for each category utilized by special need versus the regular of worker. For example, one FTE aide education program.
172
179 These data should be reported according to particularly for special education students, how people allocate their time and not it is useful to break down resource according to how their salaries are paid. teachers according to whether they are That is, an individual who is regarded as providing the traditional pull out or working in the special education program departmentalized programs as opposed to should be allocated to that program the more integrated services within the regardless of whether his/her salary is paid regular classroom. by the program. Custodial and maintenance staff have been The reader should notice the job titles or separated from food service staff to categories listed in Table 1 represent a capture the level of staffing of these somewhat expanded list from the staffing programs. Food service staff may tend to categories used in the 1993-94 SASS vary greatly among schools depending on school questionnaire.First, student the way food production services are support staff like psychologists and organized. In some cases, school lunches therapists have been separated into two are produced in one central kitchen and categories. The reason for this is to delivered to other schools, while in other capture more accurately the types of instances, each school may run its own resources being utilized in certain special cafeteria. To combine food and need versus regular programs. Therapists custodial/maintenance services would often provide direct services to students on obfuscate the different ways in which these a weekly basis, while psychologists and services are allocated and provided among social workers tend to provide services schools. more on an "as needed" basis. Reporting district administration and support Second, teachers represent the most staff by program. The increased interest in important resource in schools. For this school-level resource analysis has led to an reason, it is important to distinguish the increased demand for information on the use of teachers in primary as opposed to staffing of district administrative and supplemental assignments. The self- support functions. There is already a fair contained classroom or departmentalized level of detail about what schools look classroom teachers represent the vast like, but little is known from SASS about majority of teaching staff within a school. what district administrative structures look However, in addition, many schools like. Such data would be useful in employ resource teachers or subject area assessing what kinds of functions are specialists to provide supplemental commonly performed by district as instructional services to students.It is opposed to school-level staff.In the same important to capture such differences way SASS could request data at the school because these different types of teaching level, it could also ask for data on personnel may have fundamentally personnel at the district level who do not different impacts on student outcomes. show up on the school surveys.Staff who are assigned to schools and who provide Finally, because of the significant interest services, even part-time, on a regularly in inclusionary or integrated programs, scheduled basis at the school site should be
173 recorded at the school, while all other impact of changes in economic, personnel should be reportedas working demographic, and political trendson the out of the district office. Once again,as supply of teachers' services and thecosts with the school counts, personnel should of educational servicesacross local be counted according to FTEs and should jurisdictions and over time. be allocated among the sameprogram categories as suggested above for the In order to improve upon the analysis school-level allocation. being done in the development of models of teacher compensation, thereare three Characteristics of School and District areas in which the SASS questionnaires Personnel and Compensation need to be enhanced: themeasurement of teacher quality, the measurement of The SASS surveys have contributed benefits, and the extension of individual significantly to the analysis of teacher data to include other categories of supply and demand. These data have been personnel. Each of these is discussed used to describe in great detail the further below. characteristics of teachers, to examine those who are leaving the teaching Teacher characteristics. One of the most profession, and to analyze patterns of contentious areas in the analysis of teacher teaching assignments, to explore the supply and demand involves attemptsto factors underlying variations in teacher measure teacher quality.It is an illusive turnover, and to explain variations in concept and one that defies any simple salaries of teachers. In a current project approach to measurement. The best that for NCES, the SASS databases for 1987- one can hope to do is to obtain information 88, 1990-91, and 1993-94 are being used on a comprehensive set of teacher to construct a geographic cost index andan characteristics so that one can explore the estimate of inflationary trends in the costs patterns of variation across schools and of teachers. Development of such districts, and one can associate those geographic or time-series cost adjustments differences with variations in compensation requires detailed information on the and productivity. In 1987-88, the SASS compensation and personal characteristics teacher questionnaires requested that of teachers and the schools, districts, and individuals record the collegesor regions in which they work and live. universities attended to obtain their These indexes may be used by NCESto degrees. This question was dropped in adjust reports of nominal expenditures to 1990-91 and was restored in 1993-94. reflect real differences inresources This variable is one possible indicator of devoted to the provision of educational the intellectual capability of teachers in services. These indexes may be offeredto that it permits one to match teachers to the funding agencies at the state or federal selectivity (e.g., as measured by the level to be used in adjusting distributions average Scholastic Aptitude Test (SAT) to reflect real purchasing power of stateor scores of entering freshmen) of the federal aid to local or state jurisdictions. colleges in which they were trained. The Such indexes provide policymakers with presumption is that individuals graduating important information about the ultimate from more selective institutions of higher
174
181 education may have a greater capacity to questionnaires ask teachers what kinds of become quality teachers.It is important benefits beyond salary compensation they that this question not be dropped in the receive from the public and private school future as it is one of the few direct systems in which they teach. However, attributes of teaching personnel that is not there is no indication as to the value or explicitly recognized in public school cost of these benefits.It is important in district salary schedules.It also would be order to adequately characterize personnel useful if NCES would, as a matter of compensation that one knows how much course, obtain data on the characteristics money is contributed by the school (in the of colleges (e.g., student selectivity, case of private schools) or district (in the acceptance ratios, percent of students from case of public schools) on behalf of each the top ten percent of the graduating high employee for health and welfare insurance school class) that would be matched to this premiums, the retirement plan, and other database prior to making it available to the categories of personnel. Using this public. information, one can estimate the total cost of compensation (i.e., salaries plus the It would be preferable to obtain more cost of benefits) for each teacher. direct measures of teacher intellectual capabilities.Teachers may well be in any Conceptually the importance of fringe place in the distribution of SAT scores benefit information is to recognize that within a particular college or university. schools and school systems use a It might be worthwhile for NCES to combination of salary and benefits to explore the possibilities for obtaining attract teachers.In the long run, one information on the SAT scores of the would expect teachers to sort themselves individual teachers. One might anticipate out across districts in such a way that a that teachers would not remember their dollar of fringe benefits (in terms of the actual scores, but they might remember the cost to the district) is equal to a dollar of range within which they scored on the salary. That is, the individual would trade math and verbal components of the SATs. off salary to receive fringes up to the point This might be asked in terms of raw score at which the two have equal relative (e.g., below 400, 400 to 499, 500 to 599, marginal value in comparison to their 600-699, above 700) or percentile rankings relative cost to the district (or school).If (e.g., below 50 percentile, 50 to 75 this is true, then one should be able to add percentile, 75 to 90 percentile, or above salary costs and the costs to the district of 90 percentile). An alternative might be to fringe benefits together for purposes of explore the possibility of merging records analyzing compensation. obtained from the Education Testing Service. Much analysis has been done of variations in salaries, but only limited analyses have Fringe benefits. Data on the costs of ever been done on full compensation of benefits for various categories of personnel salaries and fringe benefits together. With would be important in improving the such information, researchers could quality of the information on examine the impact of including fringe compensation. The current SASS benefits in the analysis of teacher 175 2 compensation. This would represent a and the amount per employee is fixed for major step forward in examining the any given benefit plan.This amount can patterns of differences in teachers' salaries be represented as a fixed, lump-sum across geographic locations and between payment per employee. These health and public and private schools. Indeed, the welfare benefits include basic health data suggest that the differences in public insurance, major medical plans, dental and private school teacher compensation is insurance, vision insurance, and life larger than the differences between public insurance. To obtain estimates of the costs and private school teacher salaries because of these plans, the SASS would includea of the lower levels of benefits offered by series of questions on the public school private schools. Moreover, how does the district or private school questionnaires introduction of the benefit data affect the regarding the lump sum payments paidon patterns of variation in the costs of teacher behalf of teachers or certificated school services across different geographic employees for each category of benefit for locations? the most widely used plan (assuming the district offered multiple plans): basic Part of the reason that fringe benefit data health, major medical, dental, vision, or have not been gathered in the past is that life insurance. these data are difficult to "obtain because of the complexity of the benefit packages Another category of benefit is most across employers. Many districts offer commonly paid as a percentage of salary multiple health plans from which teachers of the employee. These benefit elements may choose and the premium paid by the include, for example, contributions by the district for these plans varies with the size employer to retirement plans (e.g., public of the employee's family. Moreover, most employee pension plans), social security individual employees probably do not payments, unemployment compensation, know exactly what the district workers' compensation plans, and contributions are without substantial effort. disability insurance. For example, the While there is probably no way to get district might contribute some fixed individual information for samples of percentage of the employee's salary to a school personnel on benefits, one could retirement program. In many states, obtain from the district-level questionnaires teachers become a part of a public in public schools and school-level employees' retirement program established questionnaires for private schools some for certificated school employees. In some information on the structure of benefits states, teachers may be part of the social that would allow researchers to estimate security system. Unemployment the value or cost of a benefit package. compensation and workers' compensation programs are most often based on the Benefits come in two basic forms: those experience of the district and rates paid by that are lump-sum payments per employee districts on behalf of employees will vary. and those that are a percentage of salary. Health and welfare benefits are most often A further complicating factor in obtaining in the form of insurance premiums paid by information on employee benefits is that in the employer on behalf of the employee, some instances, states provide for the
176 183 direct payment of benefits of public school district staff would also be extremely personnel. For example, the state of New valuable in enhancing the value of SASS York makes direct payments to teacher for analysis of the supply and demand for retirement systems for all teachers in the other critical school personnel and for state. The Department of Education in developing of geographic or inflationary Kentucky provides direct payments on cost adjustments. behalf of public school employees for many of the elements of the benefit plans. Specifically, there are two other important The problem is that district administrative categories of personnel that could be staff who are being asked to provide these covered in these samples.First, data for a district-level questionnaire may instructional aides play a potentially not have the information on the cost of important role in instructional services in a these plans per employee. number of school programs. Instructional aides are used to supplement the While benefits can be complex, they instructional program for students with nevertheless represent an important special needs (e.g., students who have component of compensation, and one on limited-English proficiencies or who are which some data can be obtained.Data eligible for special education services). from individual districts can be used to They are also used to increase the ratio of estimate benefit rates for individual adults to students in regular education employees. To illustrate one approach to classrooms. Data on samples of obtaining information on benefits, a instructional aides could provide sample set of questions is included in the policymakers with a better look at the appendix to this paper. ways in which aides are being utilized in schools, the qualifications of these Samples of other school personnel. individuals, and the patterns of Currently, SASS includes samples of compensation. individual teachers, principals, and professional library media personnel. While SASS currently gathers information These data may be used to examine issues on principals, there is virtually no related to the distribution of personal information on the high-level characteristics and abilities as well as the administrative staff (e.g., superintendents, job assignment attributes of critical deputy superintendents, and other program members of school district staff.If NCES administrators) of school systems. Who were to consider improvements in SASS are these individuals leading the nation's that would ultimately permit analyses of schools? What kinds of qualifications do educational productivity, it would be of they bring to these jobs? What factors critical importance also to include more underlie the patterns of variations in extensive information about the personal compensation of high-level administrators characteristics and job assignments of in school districts? other school and district staff who come into contact with children or who are It would also be useful to gather data on involved in leadership activities in schools samples of other noncertificated school and districts.Data on other school and 'employees. These data could be used in
177
184 conjunction with FTE counts of staff to limitations on local contributions to estimate costs at the school and district educational spending? How much does the level. The data on individual school district contribute to categoricalprograms personnel could be used to explore the for special needs children? Howare the patterns of variation in rates of pay over categorical formulas connected with time or across geographic locations and regular education funding? Are there their implications for variations in the limitations on the percentage of children costs of educational services.Currently, who can be declared eligible for certain the only such data available to researchers special needs programs (e.g., special studying these costs are from the samples education)? Are funds targeted to these maintained by the Current Population populations? Surveys.Alternatively, SASS could gather data on average salary levels or hourly Employee benefits.Information on benefit rates for those categories of personnel for programs for public employees could be which no individual data are provided. gathered to supplement the benefit information obtained at the district level. Data on State Environment in Which What contributions does the state makeon Schools Operate behalf of teachers to benefits such as health and welfare insurance or retirement As suggested earlier, SASS could enhance plans? the ability of researchers to examine the effects of state policies on resource Public school regulations. Information on allocation and outcomes by systematically the regulations under which public schools gathering data on state policies in the same operate could be gathered to allow year as the SASS questionnaires are comparisons across states of the impact on administered to local schools and school the way resources are allocated. What systems. In each year of SASS, surveys kinds of curricular standards exist within could be sent to the state departments of the state? What is the structure of teacher education to gather some of the following certification within the state? What kinds kinds of information: of laws or regulations exist with regard to collective bargaining agreements for Funding formulas. Information describing certificated employees? the structure of the school finance system in place within the state.This would Private school regulations. Information on include not only the structure of the basic the nature of private school regulations distribution formula for elementary- could be gathered. Under what kinds of secondary education, but also the nature regulations do private schools operate and structure of the categorical program within each state? Are private school funding that currently exists within the teachers required to be certified? What state. What kinds of formulas are used: kinds of safety standards are imposed on flat grants, foundation formulas, private schools? What kinds of regulation percentage equalizing, guaranteed tax base, of private school curriculum exists, if any? or district power equalizing? What do the formulas look like? Are there caps or
178
1 85 While currently there exist other data day class, a resource (pull out) program, collection besides SASS on some of these or a related service such as speech elements, it would enhance the value of therapy. For example, how much is being these data significantly if they could be spent on special education services? How linked to the SASS data collection activity. much is being spent on special day classes or resource programs and how many Data on Individual Students students receive such services?
The recent addition to SASS of However, such institutional data do not information on individual students provides indicate how these services are combined a useful structure for obtaining data on the to meet the needs of specific categories of types and costs of services received by children? They do not provide individual students across the nation. The information on the full cost of educating a current survey provides a listing of some particular type of child since most children of the classes taken by the student as well are not served within a single program. as whether or not the student is receiving For example, the vast majority of special certain other services. However, the education or other special needs children survey falls short of providing sufficient receive services in the regular program. information to estimate costs of these What is required is information on direct services. The importance of these individual children that indicates the types individual student data is that they have the of services received, the amounts of time potential to allow researchers to explore spent receiving those services, the staffing the patterns of variation in expenditures of those services, and the size of the across students with differing classes or caseloads within which those characteristics or educational needs. services are delivered.Specifically, one would need the following information to Over the years, there have been numerous enable researchers to estimate costs. attempts at the school level to determine the expenditures on programs for special Range of services received. The current needs students (e.g., special education). survey provides an overview of the types But it has been more difficult to obtain of programs in which the child participates estimates of the costs of serving a and some indication of some of the special particular type of special needs student. services received. Using a structure Data are needed at two levels. At one similar to that used to record teachers' level, policymakers want to know how schedules on the teacher questionnaire, the much is being spent to provide certain student records questionnaire of SASS types of services and programs to certain could provide for coding of all of the categories of students.This type of instructional and related service information can be obtained at the assignments of the student. The institutional level (e.g., school- or district- assignments could include a listing of all level data collections). Based on such of the secondary courses or classes as well institutional data, one can obtain estimates as elementary placements including self- of the average expenditures on specific contained classroom or resource pull out categories of services such as the special programs.
179
1R6 The student's time.For each instructional How many hours do children spend or related service assignment, the amount in school? How do they spend of time (e.g., hours or minutes per week) their time in school? the child spends in each assignment could be recorded. This information could be What kinds of courses are being used to determine: How do children spend taken by students and how do these their time in school? How much time do differ according to the educational they spend in the regular classroom? How needs of students? much time is spent in supplemental or resource programs designed for special What percentage of students are needs populations? being served in different special needs programs around the Staffing. For each instructional or related country? service assignment, one could record the types and time spent by various types of How do the ways in which these staff members while the student was children are served differ across present. For example, if the student was communities categorized by enrolled in a self-contained classroom for urbanicity, size of district, state, 30 hours per week, the respondent would and household income or poverty? record the number of hours the following staff were present in the classroom: the What are the expenditures for regular classroom teacher, a special different categories of children education resource teacher, an instructional categorized by disability or other aide, or a personal aide for this child. special need characteristics?
Class or group size.For each instructional If one were able to take the next step in or related service assignment, one could this data collection of obtaining outcomes record the size of the class or group (e.g., for individual students, then the for a pull out session) in which the child possibilities for beginning to unravel the was served. mysteries of input/output relations in education would be enhanced significantly. These kinds of data could provide valuable information on the patterns of direct Linking to Other Databases for Analysis services received by students across the country. Administrative and support A key to making NCES databases valuable services expenditures could be ascertained lies in the ability to link the various from the school- and district-level data if elements. SASS provides us with the FTE counts are obtained as suggested opportunity of gathering detailed and above.The information on direct services valuable data on the ingredients of would allow researchers to address the educational production and services as following kinds of questions: reflected in the staffing patterns across local schools and districts.But this is only
180
187 the first step.Another piece of the puzzle Burdens of Data Collection: What Do We is in linking fiscal data collection activities Give Up in SASS? to the staffing information.If the Common Core of Data (CCD) samples or The current SASS questionnaires contain a universe files could be linked with the combination of objective and subjective SASS data collection, each of the two data items. Among the objective items are activities would be enhanced. Although it counts and characteristics of staff. Among would be valuable to gather information on the subjective items are questions about the other nonpersonnel ingredients in the attitudes and perceptions of teachers, educational process through SASS, it may principals, and library media specialists. simply be creating too much in additional Perhaps SASS should focus as much as burden of data collection to obtain any possible on gathering data on the more additional information beyond the staffing objective measures of what is going on in information previously suggested. schools and less on the subjective measures. This is not to say that However, if the CCD fiscal files could be attitudinal measures are unimportant. The obtained from the schools and districts problem is that attitudinal issues are included in the SASS data collection difficult to benchmark and the data are activity, this linkage would provide difficult to interpret. What one person additional information about some of the designates as a serious problem, another nonpersonnel resources provided to person might designate as a moderate or students. Combining the staffing even a minor problem. Ultimately, information with the fiscal data on objective measures of events, activities, nonpersonnel expenditures would enhance and behaviors and their relationship the value of both the staffing and fiscal between one another, or with schooling data. outcomes, permits determination through analysis when a particular event, activity, An additional and significant step would be or behavior is a serious problem. to link SASS with data on student outcomes, either the outcomes of For example, the questionnaires ask about individual students included in the sample teacher and principal perceptions about the of students for whom the Student Records seriousness of certain kinds of problems in Questionnaire is obtained or the outcomes the school such as student tardiness, for the SASS schools. This kind of student absenteeism, teacher absenteeism, linkage would allow researchers to explore students cutting class, physical conflicts productivity issues. How do outcomes among students, and robbery or theft relate to the quantities and qualities of among many others.It may be preferable staffing in different schools? How do to obtain objective counts of events or these relationships differ in schools serving behaviors that would reveal the degree to students with exhibiting differing which students were tardy, absent, cut educational needs or economic class, or were involved in incidents or acts circumstances? of physical violence or robbery that were reported to the school office.
181
0Or The purpose of this is not to suggest that redesign of SASS with attention paid to attitudinal data are of no consequence or linking SASS with other important value.First, in the absence of more databases could be of tremendous value to objective measures of the events, the research and policy community in activities, or behaviors that these education in exploring these issues of subjective measures attempt to document, educational productivity. such subjective measures may be the only source of information. However, what is 2. Could collection of such data be being argued is that where possible, reasonably integrated into current SASS objective measures should be requested collections without increasing overall rather than information on attitudes and respondent burden? perceptions.If schools do not maintain reporting mechanisms on certain student SASS already gathers the kinds of data that behaviors or activities, then one might are important for understanding these question whether or not the activity is patterns of resource allocation. A number really a serious problem. of the suggestions made in this paper for the redesign of SASS involve asking for Second, one of the questions posed in the the same type of data in a slightly more design of this series of papers is what detailed and reorganized form. For elements in SASS could be sacrificed in example, this paper suggests order to make room for the additional data reconfiguration of the data collection on collection being proposed in this paper.In staff to be done on the basis of full-time the opinion of this author, if something equivalencies and to gather staffing needs to be sacrificed in order to make information by program of service. room for a revised data collection strategy, it is the questions on the subjective There has been some discussion at NCES perceptions and attitudes of school of gathering fiscal data at the school site. personnel. This paper suggests that it may be more efficient to gather resource data through a vehicle such as SASS where the physical Summary and Conclusions ingredients of the educational production process can be measured explicitly. 1.Could information on the source, level,, Gathering fiscal data may be more and allocation of such resources be valuable burdensome in that it requires to researchers and other data users? reconfiguration of fiscal data to match the categories (i.e., objects, functions, and With the continuing interest in issues programs) of expenditure being requested. related to educational productivity, it is Physical ingredients (e.g., FTEs) such as going to be essential for researchers and individuals by job title and program are policymakers to understand the more easily counted. Quality data on relationships between resource allocation personnel at the school level gathered and outcomes in U.S. schools. Some through SASS could reduce the need for such detailed fiscal information.
182
189 3. What would need to be given up to 5.Is this information currently collected by accommodate this increase? states, LEAs, or by other means?
It was suggested that if, indeed, something There has been some discussion within must be given up to accommodate the NCES about the possibility of moving suggested revisions or additions to SASS, toward the collection of school-level fiscal the items focused on gathering the attitudes data. The purpose of such school-level and perceptions of teachers and principals fiscal data is to obtain better information should be sacrificed.Attitudinal data are on the patterns of resource allocation as more difficult to interpret than objective they relate to issues of equity and counts of events or occurrences, and in productivity in education. But because of most instances, the attitudes are simply a the complexities of making appropriate way of trying to obtain information about adjustments in fiscal information for things that can more properly and usefully geographic or longitudinal input price be measured by objective data (e.g., differences, SASS would be a potentially student tardiness or absenteeism).If such more fruitful area of exploration for information is important, it was argued in examining equity and productivity. SASS this paper that the attitudinal data be has the potential of gathering the physical eliminated in favor of more objective ingredients rather than fiscal data that measures of events, activities, and obscures real input levels. By gathering behaviors of students or staff. data on physical ingredients, SASS avoids many of the compatibility and definitional 4.Is SASS the appropriate vehicle? issues that surround the ways in which fiscal data are reported by local What makes SASS the appropriate vehicle jurisdictions.Finally, using SASS as a for gathering resource data is that it source of resource data avoids the already focuses a significant amount of problems associated with disseminating attention on this kind of information and it and implementing a set of complex is focused at the school level. The standards for reporting fiscal information recommendations made in this paper that would be required for developing a involve modifications or additions that fit school-level fiscal data collection system. well within the existing framework and structure of the SASS questionnaires. Moreover, where expansion of the SASS structure is being recommended, it is for the purpose of expanding in important and significant ways the kinds of analyses that can be carried out using SASS. For example, it was suggested that some of the existing data collection regarding state school finance systems be integrated with the SASS data collection so that the information could be more easily linked for analytical purposes.
183
190 THE SCHOOLS AND STAFFING SURVEY FOR 1998-99: DESIGN RECOMMENDATIONS TO INFORM BROAD EDUCATION POLICY
Er ling E. Boe, University of Pennsylvania
Introduction elementary and secondary levels, and (c) changes to the content of SASS should not The Schools and Staffing Survey (SASS) increase the burden on respondents and the Teacher Followup Survey (TFS), a completing SASS questionnaires (i.e., any longitudinal component of SASS given in new content will have to be counter- the subsequent year, have been balanced by selective deletions of old administered by the National Center for content). Education Statistics (NCES), the U.S. Department of Education, during three As a contribution to NCES's current different time periods: 1987-89, 1990-92, reassessment of the design of SASS, this and 1993 -95.' Though NCES has made paper will consider future questionnaire incremental improvements in SASS' content in broad scope with respect to how between administrations and supplemented SASS can best inform education policy the basic SASS with additional issues pertaining to schools and their questionnaires addressing special topics staffing--especially important issues that such as Indian education and library may emerge over the next two decades. issues, there has been intentionally a great Specifically, the objectives of this paper deal of continuity in the content of the are to: several SASS questionnaires to permit the study of trends over the six-year period of 1. Reconsider the goals, foci, and SASS. strategy of SASS, taking into account the original framework In preparation for the next administration established in the mid-1980s and of SASS/TFS scheduled for 1998-2000, SASS's potential to inform broad NCES has decided to reassess the design education policy in the future. of SASS, including questionnaire content and related matters such as the possible 2. Identify potential new areas of linking of SASS to other NCES surveys questionnaire content related to that collect student background and schooling that are likely to be of achievement data.This reassessment is importance to education policy subject to the constraints that (a) SASS issues in the future. will continue to be a cross-sectional survey based on national probability samples, (b) 3. Review the content of the most the focus of SASS will continue to be on recent (i.e., 1993-94) SASS schools, including their staffing, at the questionnaires in light of (a) the existing balance between teacher
185
1 9 I supply and demand content and School Improvement Versus other school content, (b) the extent School Change of coverage of particular topics that appears to be excessive or the In preparation for the fourth administration coverage of topics that appears to of SASS in 1998-99, NCES is be of relatively low priority, and reconsidering the value of continuing to (c) redundancy across emphasize teacher supply and demand data questionnaires. An outcome of this in contrast with other data that may review will be the determination of contribute more toward school questionnaire content that can be improvement. Specifically, NCES asked, compressed or deleted to in commissioning this paper, "What accommodate expansion in other nationally representative schools and content areas. school process data will inform our thinking about and work toward improving 4. Recommend priorities among schools in the next 10 to 20 years" potential new content areas and (emphasis added). Given this task, essential areas from past SASS possible changes in the goals, foci, and administrations that might be strategy of SASS will be considered in included in the 1998-99 light of their potential contribution to administrations of SASS. The improving schools. focus will be on factors determining the nature and quality of schooling If the value of SASS is to be measured by that are amenable to education its contributions to school improvement, it policy interventions, as is necessary to define what is meant by distinguished from aspects of the school improvement. To begin, broader social, economic, and distinctions must be made among (a) political contexts that also shape the changes in policies that are designed to form and functions of education improve schools, (b) changes in programs within society. and practices that are designed to improve schools, and (c) changes in school Two caveats involved in the pursuit of performance reflected in indicators such as these four objectives should be noted. outcome measures (e.g., higher First, the emphasis will be on schools in achievement test scores), resource the public sector because policy formation efficiency measures, school climate in education applies predominantly to measures, and approval ratings by public schools. Second, data collected by stakeholders (e.g., parents, the public). SASS should be relevant to education Ordinarily: policy assessment and decisions at the local, state, and federal levels. Even Changes in policy are expected to though most education policy is made at result in changes in programs and the local and state levels, the mix of local, practices that conform to the new state, and federal influences on schooling or revised policies has been in gradual flux and is expected to continue to change.
186
192 Changes in programs and practices, The upshot of this line of thinking is that it in turn, are expected to result in will be more useful and productive to improved school performance translate the concept of "school improvement" to that of "school change," It is well known, however, that (a) and to determine whether changes in changes in policies (ordinarily construed policies, programs, and practices actually as, and intended to be, improvements) do lead to changes in school performance. not necessarily result in changes in Let others (policymakers, professional programs and practices, and (b) changes in educators, the press, and the public) debate programs or practices (often construed as, whether changes observed actually and intended to be, improvements) do not represent improvement. In contrast, SASS necessarily result in improved indicators of should be used to make major school performance. Therefore, the intent contributions to understanding various of policymakers and educators to improve changes in the interlocking sequence of policies, programs, and practices by policies, programs, practices, and implementing changes is not certain to performances by establishing baseline reap the desired effects on school status data and monitoring changes from performance. Whether such changes these baselines over time.' Specifically, actually affect school performance is an SASS can be used to: empirical matter, subject to systematic measurement such as by standardized Monitor changes in policy. With achievement tests. respect to policy changes, SASS has not monitored the status of, or Furthermore, there is even a problem in changes in, federal and state determining what actions or conditions education policy. However, SASS constitute "school improvement" because has and can continue to monitor the this involves judgment. What some regard status of, and changes in, policies as an improved policy, program, or at the local education agency (LEA) practice, others may view as a reversal. level through the Teacher Demand For example, some view private school and Shortage Questionnaire. One vouchers as a promising policy reform, value of monitoring the status of, while others regard them as a basic threat and changes in, LEA policies is to to public schools (Jennings & Stark, determine how policies at the 1995). In contrast, there is often federal and state levels are being consensus about what changes in school interpreted and translated into local performance represent improvement, such policy. Another value of as rising academic achievement scores. monitoring LEA policies is to Even so, judgments differ about whether determine how well programs and some changes in school performance practices at the school level represent improvement--as evidenced by conform to LEA policies. the conflict in several states over outcomes-based education. Monitor changes in programs and practices. SASS has and can continue to monitor the status of,
187 193 and changes in, programs and could have predicted in 1983 the variety practices at the LEA and school and form of such initiatives thatwere levels through the Teacher Demand taken in the decade following publication and Shortage Questionnaire and the of A Nation at Risk? Thus, it willnot be School Questionnaires. One value prudent to redesign SASS to monitor of monitoring the status of, and specific current reforms per se (suchas changes in, programs and practices systemic reform). Instead, SASS should is to determine how well they are monitor the fundamental dimensions of conforming to federal, state, and schooling that are amenable to local policies. Another value of manipulation by public policy for the monitoring the status of, and purpose of improving school performance. changes in, programs and practices is to measure many of the Considering that SASS is an instrument for dimensions of schooling that impact collecting basic data on schooling and is on school performances. scheduled to be administered only every five years in the foreseeable future, it Monitor changes in performance should be designed to cover essential, indicators. SASS has and can enduring aspects of schooling that will be continue to monitor the baseline of continuing importance, as distinguished levels of several indicators of from covering specific reforms or current school performance, and changes narrow issues. For example, if SASS from these baselines, through the monitored the character of, implementation School, Principal, and Teacher of, and changes in curriculum frameworks, Questionnaires.' Though most student assessment, and teacher school performance indicators are preparation (both pre- and inservice), it measured by instruments other than would be monitoring fundamental SASS, SASS nonetheless serves a dimensions of schooling that, as a group, special role in monitoring some are basic components of systemic reform. school performance indicators (such In the event that systemic reform, as such, as school climate) because SASS is proves to be a passing fad within a few the most direct, and possibly only, years, SASS will nonetheless have tracked source of such national data three fundamental dimensions of schooling available. that are likely to be relevant to future educational policy. Change in school policies, programs, practices, and performances is used here As noted previously, policy-based changes as a generic concept to include school in such dimensions might be viewed by improvement, reform, and restructuring, some as constructive reforms, and as since all such initiatives entail various reversals by others.Therefore, as a public forms of change. Clearly, it is not agency dedicated to collecting and possible to predict many of the school reporting unbiased statistical information, improvement, reform, or restructuring NCES should not appear to be an advocate initiatives that will continue, or be for or against any reform. Furthermore, initiated, in the coming two decades. Who the publication of major reports of
188
19 4 information based on SASS data usually Strategy: The strategy adopted for requires four years from the time of SASS has been to collect detailed questionnaire design--all the more reason descriptive data about schools and for SASS to concentrate on fundamental their staffing, as distinguished from dimensions of schooling that will be of collecting data to test specific enduring interest instead of concentrating hypotheses or to evaluate specific on any current high-profile reform policies, programs, or practices. initiative. This strategy is particularly suited to the sample survey method, and is in keeping with NCES's mission The Goals, Foci, and Strategy of to collect, analyze, and disseminate SASS: A Reconsideration education statistics.
Past SASS Goals, Foci, and Strategy It is these basic attributes of SASS that should be reconsidered in its redesign, Since its inception in 1987-88, the goals, along with the implications for foci, and strategy of SASS can be questionnaire content. characterized as follows: Future SASS Goals Goals: The primary goal of SASS has been to provide data "that will The goals of SASS should continue to be contribute to the development of the collection of data that will contribute to sound educational policies at all the development of sound education levels of government" (NCES, policies at all levels of government and 1994, p. 2). SASS has also served that will be useful to educators and as a source of national and state researchers. Changes in SASS for 1998- data about schools and their staffing 99 should be made in its foci and strategy, for educators and researchers. and in questionnaire content that follow from decisions about foci and strategy. Foci: SASS has focused primarily on the teaching force (K through Future SASS Foci 12) and secondarily on school policies, programs, and The fundamental question in redesigning SASS is whether the current primary focus administrators.Its distinctive on collecting data about teacher supply and strength has been the comprehensive data collected about demand should be maintained or modified. teacher characteristics, Major candidates for alternative foci are qualifications, and attitudes, and instruction, school governance, and school about teacher demand, supply, organization. In reconsidering the foci of turnover, and workplace conditions. SASS, the following guiding principles These aspects of the teaching force should be observed: are referred to collectively in this o paper by the expression "teacher The importance of continuity in supply and demand." data collected in previous
189
195 administrations of SASS should be Select aspects of schooling of major recognized. In light of previous public concern such as instruction investments in establishing baseline in basic skills, student discipline, data and changes over time in and school safety. numerous dimensions fundamental to schooling, continuity should be The roles and contributions of other valued highly. Therefore, the NCES surveys should be redesign of SASS should not start recognized, especially those that with a clean slate. collect data about basic dimensions of schooling and monitor change in SASS data that will be of maximum these dimensions. Even though value to policymakers and others in certain data may be of critical describing basic dimensions of importance, SASS should not schooling and in monitoring change ordinarily collect data that are over time in these dimensions available from other sources. should be of high priority.In deciding what educational These principles for reconsidering the foci phenomena are sufficiently of SASS will be used later in analyzing important to quantify, the following alternative emphases that might be guidelines can be used: employed for the design of the fourth administration of SASS. Select fundamental aspects of schooling that have been the subject Future SASS Strategy of major policy action in the recent past, such as teacher preparation In keeping with NCES's mission to and qualifications, school collect, analyze, and disseminate education accountability, decentralization of statistics, the strategy adopted for SASS authority, deregulation, has been to collect detailed descriptive data instructional technology, and the about schools and their staffing. Without like.Such topics are likely to be of challenging NCES's statutory mission, policy interest in the next decade or however, it is useful to examine three two. potential uses of descriptive data in policy development. They are: Select fundamental aspects of schooling that have been the subject Use of descriptive data for problem of recent policy analysis and definition.Since policy is formed debate, but only of minor policy in response to problems (either action to date, such as contracting actual or imagined), one policy- for school management and relevant function of descriptive data instruction (i.e., privatization), and is to quantify phenomena vouchers for private school tuition. objectively so that judgments can Some such topics are likely to be made as to whether, and to what attract major policy action in the extent, a problem exists.In this future. respect, SASS data have been
190
19 6' particularly useful in defining (i.e., addition, SASS could collect data quantifying) potential problems about the judgments of principals requiring policy intervention, such and teachers on the workability, as teacher qualifications, turnover, utility, and acceptability of policies and shortage (Bobbitt & McMillen, such as those devolving teacher 1995; Bobbitt, Leich, Whitener, & hiring decisions to the school level. Lynch, 1994). For example, SASS Data might also be collected on data have shown that the percentage teacher behavior. For example, if of teaching positions that are not it is decided to monitor a policy filled is less than 0.5 percent requiring a school improvement (Choy, Henke, Alt, Medrich, & plan every other year, data might Bobbitt, 1993). Thus, there does be collected from teachers about not appear to be a serious shortage their participation in, and of individuals available to be contributions to, such planning. appointed as teachers. However, Data thus collected about the SASS data have also shown that judgments and behavior of about 6 percent of teachers overall principals and teachers might do not hold full certification in provide insights into why some their main teaching assignment. policies seem to be implemented This can be taken as evidence of a and produce desired effects, and significant shortage of qualified why others appear not to.Finally, teachers. This problem can be some descriptive data relevant to defined even more precisely by school performance (e.g., school computing this shortage percentage attendance data, as already by main teaching assignment. collected by SASS's school questionnaires) are relevant to Use of descriptive data for evaluation evaluating policy effects--though, as of policies and programs. Without previously observed, most measures collecting evaluation research data of school performance must come specifically, descriptive statistical from sources other than SASS. data might be used by agencies other than NCES (e.g., by policy Use of descriptive data for analysts located at various enlightenment. Apart from having governmental agencies and private immediate and specific relevance to organizations at all levels) to defining policy problems and to evaluate policies and programs. evaluating policy implementation, For example, descriptive data SASS has also collected much new routinely collected on the authority background data about schools and vested in school councils (where staffing that is useful to such exist) could be used to assess policymakers and others in the extent of implementation of a fostering a broad understanding of state-wide policy to create school the phenomena addressed (i.e, the councils with authority to make "enlightenment" function of teacher hiring decisions.' In research and statistical information
191 19? as described by Weiss, 1977). "enduring issues" that are amenable to According to Shavelson (1988), the policy intervention. Enduring issues,as central value of educational well as emerging issues, will be considered research findings (and presumably because SASS may not have addressed (or the systematic collection of sufficiently addressed) either type. The statistics) lies in their ". . . second part of NCES's question, constructing, challenging, or pertaining to "how can SASS data inform changing the way policymakers and our understanding," will be addressed in a practitioners think about problems" subsequent section. (pg. 4). For example, research on teacher turnover based on SASS As addressed in this paper, the data has contributed to a better fundamental dimensions of schooling are general understanding of the classified into five main categories: school components, complexities, and governance, instruction, educational magnitude of this phenomenon finance, school infrastructure, and school (Ingersoll, Han, & Bobbitt, 1995). staffing (principals and teachers). Because the first three administrations of SASS SASS's current design emphasizes the emphasized school staffing, a subsequent problem definition and enlightenment uses section of this paper is devoted to of data collected, and minimizes SASS's consideration of this major topic. relevance to evaluation of policies and Therefore, this section of the paper programs. While the problem definition concentrates on the other four categories and enlightenment functions of SASS data (school governance, instruction, should be continued, it is recommended educational finance, and school that NCES, in the redesigned SASS, infrastructure). attempt to collect more data relevant to the evaluation function as well. Fundamental dimensions of schooling vary in the degree to which they are amenable to policy intervention. For example, the Dimensions of Schooling Amenable teacher-pupil ratio is directly amenable to to Policy Intervention policy intervention, while the social character of teacher-pupil interactions is The identification of fundamental not, even though the latter may have much dimensions of schooling that are likely to greater effect on academic achievement become the subject of policy interventions and other valued student performances. is understood here to be equivalent to Since a primary consideration in the NCES's concept of "emerging educational redesign of SASS is to inform broad issues." In commissioning this paper, education policy pertaining to schooling in NCES asked, "What emerging educational the next two decades, the objective of this issues are likely to be important in the section is to identify those fundamental coming years and how can SASS data dimensions of schooling that have a inform our understanding of these issues?" reasonable prospect of being taken This section of the paper is devoted to seriously in future policy formation. Such identifying both "emerging issues" and dimensions might be relevant to both
192
198 current policies (possibly under Because the listing in Table 1 includes too reconsideration for modification or many dimensions of schooling for practical discontinuation) and to the formation of inclusion in redesigned SASS new policy. questionnaires, a subset of these dimensions must be selected. To reduce To identify fundamental dimensions of this list, dimensions were selected that met schooling that are likely to be targets of all three of the following criteria: policy formation in the next two decades, some explicit criteria are required. Such Dimensions of schooling that are dimensions were identified here by expected to inform broad education meeting any one of the following criteria: policy (an NCES specification for this commissioned paper). Dimensions of schooling that have been the target of major policy Dimensions of schooling that can formation in the past, especially the be measured feasibly by the sample past ten years or so. survey method with strict limits on burden for respondents (e.g., it is Dimensions of schooling that have not feasible to expect LEAs to been the target of limited policy report per pupil costs disaggregated formation, but have been the target by school functions and programs of major policy attention (as because of technical difficulties and indicated by discussion, debate, burden). attempted but failed legislative efforts to adopt policies, and Dimensions of schooling that are policies subject to adoption on a not included in other high-quality small scale). surveys (e.g., the Common Core of Data includes LEA financial data). Dimensions of schooling that are currently of major concern or A listing of the subset of the dimensions contention to the public and/or meeting each of these criteria is presented policymakers. in Table 2.7 Other dimensions pertaining to teaching force are deferred to the next A listing of dimensions deemed to be section.Since each of the dimensions of fundamental to schooling because they met schooling selected for Table 2 is a one or more of the above criteria is candidate for inclusion in the redesigned presented in Table 1.6 While other SASS questionnaires for 1998-99, the observers may offer different lists, there potential of each dimension to inform probably would be a high degree of broad education policy in the next two overlap with the dimensions included here. decades will be discussed in turn.
193 199 Table 1: Fundamental Dimensions of Public SchoolingReceiving Policy Attention and/or Action***
1. School Governance/Organization 2. Instruction (continued)
Centralization/decentralization of authority "Instructional practices (continued) Regulation/deregulation Time (hours and days)/scheduling Tracking Accountability Nongraded primary levels Student outcomes, and public reporting Inclusion of special need students Competition (i.e., school choice) Inter- and intradistrict choice Instructional materials (mostly textbooks) Charter schools 'Instructional technology (computers) Vouchers (including private schools) 'Special programs Performance based accreditation Disadvantaged/at risk School performance monetary rewards Limited English proficiency 'Privatization Special education
Desegregation Community service Coordinated education, health, "School safety and discipline and social services 2. Instruction Substance abuse prevention School-to-work transition 'Curriculum frameworks/standards 'Nontraditional public schools Opportunity-to-learn standards Magnet schools 'Student performance standards Vocational education schools
'Assessment of student performance Schools-within-a-school Standardized testing Alternative schools for special Performance/authentic measurement populations Attendance Exceptional/nonconforming students Special needs students Completions/drop outs Discipline 3. Educational Finance
High school graduation standards 'Sources of school funding
"Instructional practices Per pupil cost as distributed among school functions/programs Basic skills vs. higher order thinking Course requirements 4. School Infrastructure Class size 'Building construction/rehabilitation 'Classrooms wired for computers/Internet
200 Table 2: Fundamental Dimensions of Public Schooling Recommended for Inclusion in SASS, along with Estimations of Costs of Implementing Related Education Policies
1 School Governance {Centralization/decentralization of authority} ** Regulation/deregulation *" Privatization*R
2. School Accountability School outcomes ** {Public reporting of school outcomes} *" Competition (i.e., school choice)*R {Inter- and intradistrict choice} Charter schools Vouchers (for public and private schools)
3. Standards: Curriculum and Student Performance Curriculum frameworks/standards ** Student performance standards ** Associated with curriculum standards {High School graduation standards}
4. Assessment of Student Performance Standardized testing ** Performance/authentic measurement** {Attendance} *" {Completions/drop outs} *N
5. Instructional Practices Basic skills vs. higher order thinking** Inclusion of special need students *R
6. Instructional Technology Computer usage and infrastructure ***
7. Special Programs {Disadvantaged/at risk} *** {Limited English proficiency} *** {Substance abuse prevention} **
8. Nontraditional Public Schools {Magnet schools} ** {Alternative schools for special populations}* * {Schools-within-a-school} **
Policy implementation would entail only minor incremental funding by new appropriations. Policy implementation would entail reallocation of existing funding, but little new funding. Policy implementation would require substantial incremental funding. Policy implementation would require major incremental funding. Dimensions of schooling addresed by the 1993-134 SASS. 4, School Governance governance topic of importance for monitoring by SASS data.Privatization is School-based management. In recent years, the subject of rapidly increasing policy a great deal of policy development in interest development, especially during the education has involved changes in school past year when the Republican party governance, most of which has focused on captured majorities in Congress and in school-based management--a policy most state legislatures. The two main designed to transfer authority from the forms of privatization at the present time LEA to the school level.Since major are:contracting for the management of policy interest and development on the particular schools, and contracting with locus of authority over various aspects of private organizations to operate and staff schooling are expected to continue in the charter schools--intentionally designed to future, SASS data could be very useful in have full authority and little regulation. tracking the status of, and changes in, Some vocal policy analysts and authority vested in school boards, policymakers further advocate that school superintendents, school councils, principals improvement can only be gained through and teachers. radical change in the functions of LEAs, namely, that LEAs should function onlyas Deregulation. Deregulation is another policy bodies and as contracting agencies school governance topic that has been the for public schooling. Under this subject of much policy discussion and conception, all "public" schools would be formation over the past decade or so, at operated by private corporations under both the federal and state levels. contract with LEAs, and be subject to Deregulation typically is intended to LEA policy and monitoring for contract liberate school personnel from stifling compliance. regulations so that they will be able to change school programs and practices in Privatization, in its various forms, is an ways judged to be most responsive to local appealing option to policymakers for needs and circumstances. Deregulation several reasons, one of which is that it policies have been framed in several ways, requires little or no incremental funding. such as waivers of regulations granted Instead, the costs of privatizing schools are upon application submitted by schools, largely underwritten by reallocation of deregulation for high performing schools, existing funding for regular public schools. revocation of regulations by the regulating In view of the rapidly increasing interest in authority, and the award of charter school privatization of schooling, the collection of contracts with much less regulation than SASS data about this phenomenon could applicable to regular public schools. Since be very helpful in future policy major policy interest and development in development. the area of school regulations is expected to continue in the future, it is important to School Accountability track this phenomenon with SASS data. School accountability measures, programs, Privatization. Privatization of instruction and systems have been the subject of much and school management is the third school policy development during the past decade
196
202 or so, and this is almost certain to be an continue to be widely mandated. SASS area of much policy action in the coming could collect useful data on the impact of decades. Though accountability policies this policy on the attitudes and behavior of have also been adopted for LEAs, school school personnel. Such data would be administrators, teachers, and students, the helpful in assessing whether this account- focus here is on accountability policies ability policy has the intended effect of applicable specifically to public schools. evoking school improvement efforts.
Measuring school activities. One Promoting competition.Other accountability strategy favored by accountability policies are designed to policymakers is the measurement of school promote competition among public outcomes, especially by standardized schools, and between public and private achievement tests and various approaches schools. These policies entail various to performance measurement, and also schemes for intra- and interdistrict public drop out percentages. Achievement testing school choice, charter schools, and occurs at the national level (i.e., the vouchers. School competition has been an National Assessment of Educational area of increasing policy ferment in recent Progress--NAEP) and in most states. years, is so at present, and almost SASS could be very useful in collecting certainly will be so in the future.In fact, data on (a) the types and extent of outcome charter school and voucher polices are measurements of all kinds administered in perhaps the subject of the most intense schools, (b) the time and effort the mea- policy debates at the present time. SASS suring operations consume (including can provide useful data for informing preparing students for testing), (c) the broad education policy by tracking the impact such measures have on instruction several facets of this phenomenon in terms (e.g., teaching-to-the-test), and (d) the of how it impacts on the functioning of views of school personnel on the utility of schools, on the work of their staffs, and on various types of outcome measures for school climate. improving school performance. The competitive aspects of school choice, Public reporting of school outcomes. Once charter schools, and vouchers have been states are committed to measuring school discussed here as accountability strategies outcome performances, the public used by policymakers.It should be recog- dissemination of such measures is a widely nized, however, that these strategies serve used accountability policy of very low other purposes as well. One of these incremental cost.It is popular with the purposes is to provide alternatives for press and the public who are very interest- students whose particular needs are not ed in school rankings, and this kind of being served well by instruction and other exposure brings public pressure on princi- programs offered in regular classrooms. pals and teachers to explain performance Some advocate that such students should levels attained, and to develop and report be able to choose a type of school best school improvement plans. Because public suited to them. Hence, a variety of school reporting of school performance is so choices is required for this purpose. In popular and costs so little, it is expected to addition to competition, privatization of
197
203 schooling serves a further function. As policy attention in recent years. They noted above under school governance, the represent the level of attainment expected charter school and voucher varieties of by the time of completion of secondary school competition are also strategies used school, and are relevant to school-to-work to remove schools from the direct control and school-to-college transitions. of LEA's and to place them under private auspices. Assessment of Student Performance
Standards: Curriculum and Student The assessment of actual student Performance performance (as distinguished from expected standards of performance) isa Curriculum. The development of major ongoing subject of policy formation curriculum standards represents a major and is central both to instruction andto current policy initiative at the federal, school performance. SASS can have two state, and local levels, even though there important roles in tracking the assessment has been considerable conflict over of student performance. One role is to standards developed in some subject survey the types of measures used for matters. Because the development of academic achievement, especially standards represents a major policy trend standardized and performance/authentic that lies at the center of the teaching and measurement. However, it is not expected learning process (as distinguished from, that SASS will attempt to collect dataon for example, school governance), it would the results of such measures due to the be worthwhile to track the types and burden involved and because these results sources of curriculum standards used in are often available from state education the nation's schools. agencies. The second role for SASS is to collect data on the level of student Student performance. Even though the performance based on other types of development of student performance indicators (e.g., attendance data) because standards trails behind the development of these data are important and the burden is curriculum standards, it would also be modest. worthwhile to track whether student performance standards are used, and, if Instructional Practices so, their source and whether they are correlated with curriculum standards that As noted above, NAEP surveys address may be in use. instructional practices in some detail. Therefore, little in this category is Opportunity-to-learn standards are omitted recommended for SASS coverage except here because NAEP surveys already for two instructional practices that are not address instructional practices in some covered by NAEP, have been contentious detail. with educators and parents, and have been the subject of continuing policy attention. High school graduation standards are The first of these two practices is the recommended for inclusion in SASS instructional emphasis placed on basic because they have also been the target of skills in contrast with the emphasison
198
204 higher order thinking. The second problems in schools, and the types and instructional practice is the inclusion of extent of school programs designed to "special needs" students in regular address them. classrooms. If SASS data are collected on these two practices, it would also be useful Nontraditional Public Schools to survey the type and amount of instructional emphasis given to four other Nontraditional schools (e.g., magnet topics that have been of interest in the schools, schools-within-schools, alternative development of policy, namely, discipline, schools, and specialized schools for working together cooperatively, values, students with severe disabilities) have been and computer literacy. established in response to a variety of social and human concerns such as Instructional Technology desegregation, school drop outs, alienation, and learning and physical Many policymakers and others expect handicaps. Since the concerns to which microcomputers in the classroom to these schools respond will continue in the revolutionize instruction, and continuing decades to come (and may even intensify), policy attention to this topic is expected. SASS data would be useful in informing SASS can inform policy development in policy review and development in this this area by collecting data about the area. availability in classrooms of computer hardware, networking and access to the The strategy to break up very large (and, Internet, and software for instructional therefore, impersonal) schools into smaller purposes, how and the extent to which it is schools (i.e., schools-within-a-school) is an used, and other important aspects of this instance of a larger strategy designed to technology (e.g., the availability in schools improve schools. Data that are collected of skilled technicians to install and routinely by SASS on school size and type maintain this technology). are useful for tracking changes in these dimensions. Special Programs
A number of special programs have been Review of Prior SASS implemented in public schools to address a Questionnaires variety of social problems (especially those of poverty, limited English proficiency, Improvements in, and changes to, the and substance abuse) that limit student contents of SASS questionnaires have been performance. These concerns are expected made by NCES for the 1990-91 and 1993- to continue to command the attention of 94 administrations. There was substantial policymakers, educators, and the public in deletion of content from the 1990-91 coming decades--especially since current survey and addition of new content for the trends suggest that such social problems 1993-94 survey. Many of the items are intensifying. SASS can contribute to deleted from the 1990-91 survey pertained policy development in this area by to teacher supply and demand. Some of collecting data about the extent of such these changes also involved the deletion of
199
205 items from the school questionnaires that maintain SASS in its 1993-94 form. Past duplicated similar items from the teacher decisions to emphasize collection of data questionnaires. Nonetheless, in about teachers and principalswere astute commissioning this paper, NCES asked, and justified because staffing is the central "Is the existing balance between supply factor in determining the quality and and demand issues and other school topics improvement of schooling. This view is still appropriate?" The review of SASS buttressed by the results of a recent large- questionnaires presented here will yield an scale meta-analysis of education production answer to this question about priorities. functions that found that "resource variables that attempt to describe the Argument for improving SASS. As in the quality of the teachers (teacher ability, past, NCES can easily justify making teacher education, and teacher experience) incremental improvements in, and changes show very strong relations with student to, SASS. That is not the issue. The achievement" (Lain, Greenwald, & Hedg- issue is whether the content of SASS es, 1995, pp. 57-58).It can further be questionnaires for 1998-99 should argued that, since the three past SASS represent a drastic change from the past--a administrations have yielded a large change characterized by major reductions amount of data basic to understanding the in content pertaining primarily to teacher dynamics of the teaching force (including supply and demand (but also to principal trends over time), it is vital that continuity supply and demand), and the addition of in data collection be maintained about this new content about other dimensions of most important component of the quality schooling that would better inform broad and character of schooling. education policy. Recommended approach. In response to the The argument for changing SASS genuine tension between the strategies of drastically is that the first three rounds "drastic change versus continuity" in have yielded substantial data about the redesigning SASS for 1998-99, thispaper teaching force and additional data are takes a middle position. While a consider- unnecessary. Also, that by continuing to able amount of the content of the 1993-94 emphasize teacher variables, an SASS questionnaires can be compressedor opportunity is lost to collect data about deleted to accommodate expansion in other aspects of schooling that will be of content, the first priority is to maintain importance to future policy development continuity in data collection basic to on a wide range of other issues. understanding the attributes and flows of the teaching force. The basic data Argument for not modifying SASS. On the collection to be maintained for this other hand, an argument can be made to purpose is presented in Table 3.
200 206 Table 3: Fundamental Dimensions of the Teaching Force Addressed by SASS Questionnaires
1. Teacher demand 9.Turnover 'Among schools 2. Sources of supply Among teaching fields 3. Teacher shortage Between sectors 4. Teacher demographic characteristics Attrition
5. Teacher qualifications 10. Compensation 'Teacher preparation 'Level of compensation Preservice 'Minimum compensation Professional development Special monetary incentives 'Certifications Salary setting principles Experience Salary schedules 6. Teaching assignment and load Merit pay Career ladder 7. Employment and working conditions Employee benefits 8. Collective bargaining/union membership
Although it is widely recognized that conditions. Although teaching teacher quality is perhaps the most critical performance is the prime facet of teacher educational determinant of student quality, a promising research approach to achievement (Kennedy, 1992; Mandel, investigating this topic (as distinguished 1995; Laine, Greenwald, & Hedges, from collecting survey data) has been 1995), teacher quality is a broad concept recommended elsewhere (Mandel, 1995). that includes (a) teacher qualifications, (b) classroom teaching performance, and (c) As to teacher tested ability, NCES should teacher ability such as measured by tests. study (if it has not already done so) the The sample survey method is very useful feasibility of either (a) collecting tested for collecting data on teacher ability scores for teachers in the SASS qualifications, and SASS should continue sample, or (b) linking SASS to other to have a strong emphasis on collecting databases where such scores might be such data as it has in the past. SASS is recorded. Even tested ability scores for a not a suitable vehicle for collecting data on subsample of the SASS sample of teachers the quality of classroom teaching would be useful. performance, even under simulated
201
2 0 7 Teacher qualifications are also directly will be helpful in deciding what content to relevant to the dimension of teacher retain, condense, or delete: shortage, as listed in Table 3.Information derived from SASS data demonstrates that Content basic to understanding the there is practically no shortage of the characteristics, qualifications, numbers of individuals that are willing and responsibilities and authority, able to accept teaching positions (Choy et leadership style, and compensation al., 1993).Instead, other research has of school principals should be shown that there is a shortage of qualified retained, but be reduced in depth of teachers, and an even greater shortage of coverage. high-quality teachers (Boe & Gilford, 1992; Gilford & Tenenbaum, 1990). Content should be retained that has Therefore, the continued collection of the potential to inform broad extensive data on teacher qualifications by education policy in the coming two SASS will be vital to measuring the level decades, while content that does not of, and trends in, the shortage of qualified should be minimized. The topics teachers. listed in Tables 2 and 3 suggest what this content should be. As in past administrations of SASS, it is recommended that data pertaining to Content should be minimized that dimensions of the teaching force listed in is, or will be, available from other Table 3 should continue to be collected high-quality surveys. through the teacher demand and shortage, school, and teacher questionnaires. Content should be minimized that However, the breadth of coverage of these has shown a stable pattern over the dimensions should be reduced to a first three SASS administrations if reasonable minimum. Some specific it is likely to remain stable or can suggestions for minimizing the breadth of be predicted with reasonable coverage of some of these dimensions are accuracy. made below. One general recommendation is to eliminate all Content should be minimized that redundant item content that may still exist has generated little policy or between questionnaires and within research interest in the past, unless questionnaires.' there is reason to believe that it will become useful to emerging With respect to the remaining content of policy issues. SASS questionnaires for 1993-94 (i.e., other than content relevant to the Content should be minimized that is dimensions of the teaching force identified based on questionnaire items of in Table 3), much should be condensed or marginal technical quality (unless deleted to accommodate new content in efforts to improve the items are 1998-99 that will better inform broad successful). education policy. The following guidelines
202 208 Content of Main SASS Ouestionnaires but consideration might be given to casting it in terms of decision-making authority. Based on the general considerations discussed above for redesigning SASS School questionnaires. Much of the content questionnaires, specific observations and devoted to the teaching force in the first suggestions are made below relating to the two administrations of the school content of each of the four main SASS questionnaires was eliminated for the third questionnaires used in 1993-94. administration in 1993-94. As last designed, the content of the school Teacher demand and shortage questionnaire. questionnaires was focused on basic The Teacher Demand and Shortage descriptive information about school Questionnaire is the only source of some characteristics, staffing, programs, and teacher data and should continue to be policies. Much of this is essential to collected from LEAs. Items such as describing schools, and much is relevant to district teacher counts, hiring criteria, and educational policy issues.Therefore, it collective bargaining agreements should be would probably not be prudent to delete or retained.In contrast, items pertaining to compress much of this material. teacher type, certification, supply, turnover, ethnicity, and retirement benefits Teacher questionnaires. The content of the are candidates for deletion, because such teacher questionnaires needs to be trimmed information is included in the Public to permit the inclusion of expanded content School Teacher Questionnaire. Much of relevant to education policies in areas the other content of the Teacher Demand other than the teaching force. Some and Shortage Questionnaire addresses the suggestions are: (a) compress the content dimensions of schooling listed in Table 2, on teacher experience (especially breaks in and are, therefore, candidates for service and experience prior to beginning retention. teaching), (b) eliminate content on changes in teaching assignment (as data on year-to- Principal questionnaires. Though collection year changes are available from the TFS), of data on the education and experience of (c) compress the content on teaching load, principals is important and should and (d) compress or eliminate much of the continue, it would seem that assigning content on teacher perceptions and two-thirds of the content of the question- attitudes (depending upon the extent to naire to these two topics is excessive, and, which previous analyses have demonstrated therefore, should be condensed. While the stability in data pertaining to these topics, item pertaining to school problems may and the extent to which these data have continue to be important, it is doubtful that proven to be interesting or useful to the same item should continue to be policymakers and others). While these and included in both the principal and teacher other changes might be made to data questionnaires. The content on locus of collected from teachers, it is important to influence is relevant to governance policies continue to collect sufficient data to promoting decentralization.Collection of monitor all the dimensions of the teaching data on this general topic should continue, force listed in Table 3.
203
9 0 9 Teacher Followup Survey Discussion and Recommendations As a longitudinal component of SASS, the TFS has served an indispensable role in Feasibility of Expanding SASS Content monitoring year-to-year flows of teachers included in the prior SASS sample. Asa To recapitulate, it is recommended that vehicle for tracking actual teacher career future data collection by SASS be transitions (as distinguished from teacher prioritized as follows: reports of activities in the prior year, and plans for the coming year), TFS is the 1. Include fundamental dimensions of definitive means for collecting data on the teaching force as listed in Table teacher turnover and variables associated 3, with emphasis on teacher .with teacher turnover.Therefore, TFS qualifications. should be retained in much the same form as in the past. 2. Include basic attributes of school principals, LEAs, and schools. Linking SASS with Student Data 3. Include fundamental dimensions of Because data collected by SASS provides schooling that are expected to be critical national- and state-level informa- active areas of policy development tion about schools and their staffing that is in the next two decades, as listed in unavailable from others sources, SASS Table 2. most definitely should be continued in its present general form. Yet, a major limitation of SASS is that it does not This is to be accomplished without include student achievement data thatcan increasing the burden on SASS be analyzed in relation to school, teacher, respondents. and principal variables.Fortunately, NCES is exploring the possibility of In contemplating the feasibility of adopting linking SASS to student data collected by these recommendations, it should be other surveys such as NAEP and the recognized that the four SASS National Eduction Longitudinal Survey of questionnaires used in 1993-94 already 1988 (NELS:88). The advantages of addressed a substantial majority of the doing so are obvious if the important role dimensions recommended here. The only served by SASS in NCES's current array new topics recommended for inclusion in of surveys is not greatly diluted or the next SASS are some of the dimensions sacrificed. If such radical changes in of schooling listed in Table 2. Ascan be SASS were required to link it with student seen in this table, the 1993-94 SASS data that much of its current value to the collected data pertaining to half of the field would be lost, then other solutions dimensions listed. SASS has also collected should be sought (e.g., expanding or data on all the teacher, principal, and otherwise changing the teacher and school variables recommended above for school/school administrator questionnaires continued coverage in the 1998-99 SASS. of NAEP and NELS:88). To offset the burden created by adding
204 210 new content, other recommendations were schooling is related to the "means" made about compressing or deleting available to policymakers to influence the content from the four SASS questionnaires educational process.Specifically, used in 1993-94 (see section V). policymakers can influence education by the following means.9 It, therefore, seems feasible to consider redesigning SASS in accordance with the They can structure the governance recommendations offered in this paper. context by: The fact, however, that past SASS questionnaire content included half the Allocating decision-making dimensions of schooling listed in Table 2 authority (such as in school-based does not imply that their coverage was management). adequate (either in form or breadth) for future purposes. Therefore, it may be Adopting or revoking mandates necessary to establish further priorities for (i.e., in the forms of statutes and selecting among the specific dimensions of regulations) (such as deregulation). schooling recommended in Table 2 for future data collection, as addressed in the Designating public or private next section. sector for operations (such as in privatization). Data Collection Priorities for Dimensions of Schooling They can establish accountability policies, with performance As noted above, a number of the incentives, designed to: dimensions of schooling listed in Table 2 represent new content areas recommended Measure and report school for data collection by SASS because of performance (such as achievement their potential relevance to future policy test scores). development.If it is not feasible to collect SASS data about all these dimensions, then Promote competition among the subset selected should include the schools (through school choice, dimensions that are likely to be the most charter schools, vouchers). active areas of policy debate and development. The best candidates for -- Link rewards and sanctions to areas of most intense policy development school performance (such as are school governance and accountability- offering monetary rewards). just as they have been in the recent past. It is, therefore, worth examining why They can appropriate funds by: policymakers have attended so extensively to aspects of school governance and Making direct allocation of funds accountability. for programs, and to build capacity in terms of human, equipment, or A primary reason for the attention given infrastructure resources (such as by policymakers to these two facets of computer acquisitions).
205
211 -- Creating financial incentives to only minor or moderate incremental evoke desired responses (such as funding is required to implement policies school improvements). in the governance areas. Furthermore, policy decisions about school governance Adopting governance and accountability can often be translated into action much policies is appealing to policymakers for more quickly than can changes in several reasons: (a) the funding required is instruction.Consider, for example, the relatively low, (b) implementation can relative ease with which regulationscan be often be accomplished quickly by central repealed in comparison with implementing action without the delay and difficulty curriculum standards in the classroom. involved in securing the participation of These characteristics of schoolgovernance professional educators (i.e., district policies help explain why they have been superintendents, principals, and teachers), so prevalent in recent years, and why they (c) policymakers appear to be decisive and are expected to be a major area of policy to have educational problems under development in the coming decade or two. control, and (d) the policies often enjoy A high priority should be placed by NCES broad public support. For all these on collecting SASS data about school reasons, implementation of governance and governance arrangements in the interest of accountability policies represents good informing broad education policy. politics.Therefore, such policies have been quite popular with policymakers for Policymakers also influence schooling by at least the past decade, and probably will establishing accountability procedures and continue to be so in the foreseeable future. systems applicable to LEAs, schools, In the following paragraphs, the three teachers, administrators, and students. means available to policymakers to Such policies are popular with the press influence the educational process are and with the public because they are taken discussed in greater detail. as evidence of engagement, oversight, and control by responsible authorities. As Through structuring the governance indicated in Table 2, a remarkable variety context of schools, policymakers can of school accountability strategies are control the governance environment in available to policymakers and many of which schools operate by (a) assigning these have been used widely (Boe, Boruch, decision-making authority along the Landau, & Richardson, 1993), while only continuum from centralization to minor or moderate incremental funding is decentralization (the main levels of the required to implement them. In addition, continuum being Federal, state, LEA, and such policies can often be implemented school), (b) adopting or revoking statutes quickly and easily since they do not and regulations (both as to the content and require the collaboration of school the general extent of control of schooling), personnel. These circumstances explain and (c) designating the functions of why the development of accountability schooling that will be performed by policies has been so prevalent in the past agencies in the public and private sectors decade or so, and why it is reasonable to (such as privatizing the management of expect that this will continue to be a very public schools). As indicated in Table 2, active area of policy development in the
206
2 12 future.It is, therefore, recommended that changes in school programs and practices- NCES place high priority on collecting both time-consuming processes. For all data about various forms of school these reasons, progress in improving accountability. instruction is likely to be slow and uneven- -even though central to improving teaching The appropriation of funds to improve and learning.Regardless, it can be schooling is, of course, a means also used expected that policymakers will continue to extensively by policymakers. As noted devote considerable attention to above, one of the main advantages of instructional issues in the coming decades, developing policies of the school and SASS data can be very helpful in governance and accountability types is that informing broad education policy in this they generally do not require substantial area as well. funds for implementation. However, the appropriation of substantial or The considerations discussed above suggest major funds is usually required to that policy development in the areas of implement policies intended to improve school governance and accountability will instruction such as curriculum and be particularly active (and implementation performance standards, educational of such policies will be feasible) during the technology, and special programs such as next two decades, while policy for at-risk students (see items 3 through 8 development in the area of instruction will of Table 2). At this time of retrenchment continue to be fraught with great in government spending, tight finances difficulty.Therefore, if priorities need to alone represent a major impediment to be established for SASS data collection rapid progress in improving instruction. among these three areas, it is recommended that priority should be With respect to adopting policies intended accorded to school governance and to improve instruction, several factors accountability in the interests of informing other than funding limitations also pose policy deliberations. serious difficulties to implementing such policies. For example, efforts to Evaluation Function of SASS Data implement curriculum, opportunity-to- learn, and performance standards have To enhance the value of SASS data for become embroiled in controversy over the policy development, it has been proper role of federal and state policy in recommended above that SASS this area.Policies addressing other questionnaires be designed to collect more instructional issues (such as authentic data that is useful in evaluating policies, assessment, inclusion of special needs programs, and practices. While NCES students in regular classrooms, programs should maintain a neutral posture with for limited English proficient students, and respect to the import of such data to policy so on) are also very controversial. In issues, it seems feasible for NCES to addition, implementation of policies collect and report descriptive statistical intended to improve instruction usually data of this type. In fact, such data were involve (a) changes in the work of collected in the 1993-94 SASS, as principals, teachers, and students, and (b) illustrated by an item in the teacher
207
213 questionnaires which sought teachers' be useful for assessing whether the opinions about the impact and value of program or practice embodies the professional development programs in basic intent of the policy. For which they had recently participated. example, information could be However, an item of this type represents collected about school-level efforts only one of four types of descriptive data to implement a school-based that could be very useful for assessing management and the specific form policies and programs. The several types taken by this management of data relevant to evaluation that might be arrangement. collected through SASS are: 3. Data on judgments by principals and 1. Data on policies adopted. Data on teachers. In those instances where whether an LEA has adopted a a policy has been implemented at policy locally can be useful in the school level, the judgments of evaluating the acceptability or principals and teachers can be feasibility of a policy originating at collected about the workability, a higher level.For example, a utility, and acceptability of the new state may promote (say, through programs or practices that result financial inducements and provision from implementing the policy. For of technical assistance) the example, the judgements of voluntary adoption of school-based principals and teachers on various management by LEAs. The aspects of a school-based incidence of adoption of the new management system implemented in policy would be relevant to their school could be useful in assessing the strategy used by the assessing the merits and liabilities state to promote this policy. of delegating operational authority to the school level. 2. Data on policies implemented as programs or practices.In those 4. Data on behavior of principals and instances where an LEA has teachers. In those instances where adopted a policy, data on whether it a policy has been implemented at has been implemented at the school the school level, reports by level in the form of a program or principals and teachers could be practice also can be useful in collected about changes in their evaluating the acceptability or behaviors that have occurred as a feasibility of a policy.If a policy result of implementing a policy. has not been implemented, further For example, the work of data can be collected on barriers to principals is expected to be changed policy implementation; if a policy substantially by the introduction of has been implemented as a program school-based management. If the or practice, further data can be workload has shifted, for example, collected that describe the program from 50 percent instructional or practice, and any unexpected support and 10 percent financial side effects. Such information can management (plus other functions),
208 214 to 10 percent instructional support and accountability listed in Table 2, rather and 50 percent financial manage- than all the dimensions relating to ment, such information would be improvement of instruction. useful to policymakers and others to assess the policy impact on the The guidelines that were used in this paper culture of schools. to select the dimensions listed in Tables 2 and 3 were: The discussion above demonstrates that descriptive statistical data such as collected Select fundamental dimensions of by SASS could be very useful in assessing, schooling, especially of public refining, and developing education policy. schools If the evaluation function of SASS is construed in this way, it represents an Emphasize the selection of approach to securing the systematic (as dimensions amenable to policy distinguished from anecdotal) input of intervention at all levels principals and teachers on initiatives that have been taken, or might be taken, to Emphasize the selection of improve schooling. Policymakers and dimensions pertaining to the others would then have sound information teaching force because of the about the views of principals and teachers central role of teachers in the about what is workable and useful, and quality and improvement of what is not.Therefore, the evaluation schooling, and in the interest of function of SASS data should be enhanced. continuity with past SASS administrations
Summary Emphasize the selection of dimensions of schooling that have The final question posed by NCES in the greatest potential to be policy commissioning this paper was: "What are issues during the next two decades the likely concerns of the next 10-20 years, and what data should we collect Include dimensions of major now to inform those issues in the next concern to stakeholders decade?" The short answer to the first part of this question is that SASS should Exclude dimensions for which collect data relevant to the fundamental nationally representative data are dimensions of public schooling listed in collected in other high-quality Table 2 and the dimensions of the teaching surveys force listed in Table 3, for reasons previously discussed.If it is not feasible Exclude dimensions that are to collect data on all the dimensions listed inappropriate for questionnaire in Table 2, then it is recommended that the surveys priorities for expanding SASS questionnaire content should include the Exclude dimensions that pose an various dimensions of school governance unacceptable burden on respondents
209
21 After thus having selected the sets of recommended that questionnaires be dimensions listed in Tables 2 and 3 for designed to continue to collect useful data inclusion in SASS questionnaires, the for these purposes, and, in addition, be second part of NCES's question can be designed to collect data more useful for addressed, namely, "What data should we evaluation of policies and programs. collect now to inform those issues in the next decade?" In summary, it was recommended that: References
Data should be collected to quantify Bobbitt, S. A., Leich, M. C., Whitener, the baseline status of schools and S. D., & Lynch, H. F.(1994). their staffs with respect to the Characteristics of stayers, movers, and dimensions selected for inclusion in leavers:Results from the Teacher SASS questionnaires Followup Survey, 1991-92 (NCES 94- 337). Washington, DC: U.S. Department Once baseline status is established, of Education, National Center for subsequent administrations of SASS Education Statistics. should monitor possible changes from baseline status in the Bobbitt, S. A., & McMillen, M. M. dimensions of interest (1995).Qualifications of the public school teacher workforce: 1988 and 1991 (NCES The collection of data on both 95-665). Washington, DC: U.S. baseline status and changes from Department of Education, National Center baseline should be designed so that for Education Statistics. the data are useful to inform education policy development. The Boe, E. E., Boruch, R. F., Landau, R. value of such data for policy E., & Richardson, J. A. (1993).State development will be maximized if policies fostering the entrepreneurial the data are: restructuring of public schools: Identification and classification based on a Useful for problem definition, survey of the fifty states (Research Rep. -- Useful for evaluation of policies No. 1993-ER1). Philadelphia, PA: and programs, and University of Pennsylvania, Graduate Useful for enlightenment School of Education, Center for Research and Evaluation in Social Policy. In the past, SASS data have been especially useful for problem definition Boe, E. E., & Gilford, D. M. (Eds.). and for enlightenment. In the future, it is (1992). Teacher supply, demand, and quality: Policy issues, models, and data bases. Washington, D.C.: National Academy Press.
210
1 6 Choy, S. P., Henke, R. R., Alt, M. N., Laine, R. D., Greenwald, R., & Hedges, Medrich, E. A., & Bobbitt, S. A. (1993). L. V. (1995). Money does matter: A Schools and staffing in the United States: research synthesis of a new universe of A statistical profile, 1990-91 (NCES 93- education production function studies. In 146). Washington, DC: U.S. Department L. 0. Picus & J. L. Wattenbarger (Eds.), of Education, National Center for Where does the money go? Resource Education Statistics. allocation in elementary and secondary schools (pp. 44-70). Thousand Oaks, CA: Gilford, D. M., & Tenenbaum, E. (Eds.). Corwin Press. (1990). Precollege science and mathematics teachers: Monitoring supply, Mandel, D. R. (1996). Teacher demand, and quality. Washington, DC: education, training and staff development: National Academy Press. Implications for national surveys.In G. Hoachlander (Ed.), From data to Ingersoll, R. M., Han, M., & Bobbitt, S. information: New directions for the (1995). Teacher supply, teacher National Center for Education Statistics qualifications, and teacher turnover:1990- (NCES 96-901). Washington, DC: U.S. 91 (NCES 95-744). Washington, DC: Department of Education, National Center U.S. Department of Education, National for Education Statistics. Center for Education Statistics. McDonnell, L. M., & Elmore, R. F. Jennings, J. F., & Stark, D. (1996). (1987). Getting the job done: Alternative Tracking education reform: What type of policy instruments. Educational national data should be collected through Evaluation and Policy Analysis, 9, 133- 2010? In G. Hoachlander (Ed.), From 152. data to information: New directions for the National Center for Education Statistics Shavelson, R. L.(1988). Contributions (NCES 96-901). Washington, DC: U.S. of educational research to policy and Department of Education, National Center practice: Constructing, challenging, for Education Statistics. changing cognition.Educational Researcher, 17(7), 4-11. Kennedy, M. M. (1992). The problem of improving teacher quality while balancing Weiss, C. H. (1977). Research for supply and demand. In E. E. Boe & D. policy's sake: The enlightenment function M. Gilford (Eds.), Teacher supply, of social research. Policy Analysis, 3, demand, and quality: Policy issues, 531-545. models, and data bases (pp. 117-122). Washington, DC: National Academy Press.
211 217 1. This paper assumes that the reader is familiar with background information about SASS. Inbrief, SASS is a large-scale cross-sectional survey with different questionnaires being administeredto independent national probability samples of local education agencies (LEAs), schools, principals, and teachers in the public sector (variations of the questionnaires for schools, principals, and teacherswere administered in private and Indian schools).In the public sector, schools are sampled first, and teachers are sampled within the schools.In addition, the principals for the sampled schools and the LEAs in which the schools are nested are included. Thus,responses to the several questionnaires can be linked. During the year following a SASS administration, the Teacher Followup Survey (TFS) is administered to three subsamples of teachers as follows: (a) teachers who continued to teach in the same school as in the SASS year, (b) teachers who transferred to a different school in year after SASS, and (c) teachers who left the teaching profession at the end of the SASSyear.Descriptive information about SASS and TFS is available from NCES.
2. Throughout this paper, references to SASS alone imply SASS and TFS.
3. In fact, baseline status data may often be of more value in informing broad education policy than data on changes from baseline. Therefore, when this paper discusses changes in school policy, programs, practices, and performance, it should be understood that establishing baseline status data isa necessary and integral part of measuring change.
4. The major examples are school attendance items in the School Questionnaires, and school climate in the Principal and Teacher Questionnaires.
5. The Public School Questionnaire of SASS's third round already collects information about several functions of school councils.
6. Other than the dimensions of school listed in Table 1, SASS has also collected data on a variety of basic aspects of schooling such as student enrollment in LEAs and schools, staffing pattern and size, school type and level, location, etc., and should continue to do so.
7. It is possible that data for some dimensions included in Table 2 are available from other NCES surveys and should, therefore, be deleted from this list.This can best be determined by NCES staff members who know the detailed content of all their surveys.
8. It is recognized that most or all of such redundancies were eliminated in the 1993-94 SASS.
9. The following outline is based in part on the identification of policy instruments by McDonnell and Elmore (1987).
212
218 1998-99 SCHOOLS AND STAFFING SURVEY: ISSUES RELATED TO SURVEY DEPTH
Susan P. Choy, MPR Associates, line.
Introduction (2) From whom should data be collected? Should any new The Schools and Staffing Survey (SASS), categories of respondents be added first fielded in 1987-88, represented a to the survey? major redesign of the NCES elementary and secondary data collection, and it has (3) How much data can reasonably be proved to be a successful one. collected from each respondent? Thoughtfully designed and competently What options are available to executed, the SASS linked surveys have expand data collection without made it possible to develop comprehensive overburdening respondents? profiles of the nation's public and private schools and teachers and principals, and to The paper starts by describing the context examine supply and demand issues more within which choices about survey depth thoroughly than ever before.Successive must be made.It examines the purpose of administrations of SASS in 1990-91 and SASS, identifies some important emerging 1993-94 brought improvements in sample policy issues and their implications for the and questionnaire design and supplied survey, and describes the users and uses of additional information on timely topics. the data. The paper then takes each set of The switch to a five-year cycle of data questions in turn, discussing the issues collection provides some time to they raise and making specific reexamine the survey's design and recommendations on how to address them. consider changes to enhance the relevance and quality of the SASS data collection effort. Context for Considering Survey Depth This paper focuses on one aspect of the survey's designsurvey depthand To make appropriate decisions about addresses three major questions: survey depth, we must first consider the purpose of the survey, who uses (or might (1) At what level should estimates be use) it, and to what end. Although the provided? Are state- and affiliation- future scope of SASS is currently being level estimates useful enough to debated, some assumptions were necessary justify the large sample size they in order to make recommendations about require? Are the oversampled survey depth. For the purposes of this groups of schools and teachers still paper, I have assumed that the general important to single out? goals and structure of the survey will remain intact, but have also taken into
213
219 account emerging policy issues that might of schools and subject areas using various appropriately be monitored through SASS. definitions of "qualified," the match Thus, although this paper focuses on between qualifications and assignments as survey depth, it unavoidably strays into the indicators of shortages in various fields, realm of survey scope. difficulties filling new positions, the adequacy of the supply of minority Purpose of the Schools and Staffing Survey teachers, and the relationship of school and teacher characteristics to attrition. SASS was designed to inform Congress, the Department of Education and other Although teacher shortages may not exist federal agencies, state education agencies, in the aggregate now, monitoring supply local districts, educational associations, and demand remains a worthy goal and the larger education community on because shortages could still become a four major topics: (1) teacher supply and problem in the future.In 1993-94, about demand; (2) school conditions and one-quarter of all newly hired teachers programs; (3) the characteristics of the were not teaching the year before (Henke, elementary and secondary teaching force; Choy, & Geis, 1996). Thus, the size of and (4) the characteristics of principals the pool of potential teachers is uncertain (National Center for Education Statistics and may be influenced by many factors [NCES], 1995b). In the mid-1980s, when such as teacher salaries, the attractiveness the first SASS was designed, many of alternative careers, and the general state analysts were predicting a serious of the economy, to name a few. impending teacher shortage, particularly in math and science. An important purpose Like supply and demand, school conditions of SASS, therefore, was to monitor the and programs and the characteristics of the flow of teachers nationwide as they moved school work force still command attention, in and out of the teaching work force in although the focus of some of the ways that earlier elementary and secondary questions has changed in these areas as data collections were unable to do. well. SASS has been updated to reflect this. Compared with previous After more than a decade of concentrated administrations of the survey, the 1993-94 attention to school reform at the federal, SASS has more questions on teacher state, and local levels, the four general education and certification, professional topics mentioned above remain as relevant development, the influence of various as ever. The focus of some of the participants in school decision making, and questions related to them has changed, the variety of programs and services however. For example, when the schools offer. predicted teacher shortages did not materialize, researchers and policymakers In summary, SASS has been well shifted the emphasis of their work from structured to capture information on the absolute number of teachers to more enduring issues and has been modified complex issues such as the supply of appropriately to reflect changing policy teachers in specific fields, the distribution concerns through the early 1990s. Now, of qualified teachers across various types however, after more than ten years of
214 reform initiatives, a hard look at the How students are prepared for information being gathered is necessary; it school (preschool education, early is important to monitor the implementation childhood experiences), and how and impact of these initiatives, and to try they are assisted in making the to anticipate what people will want to transition from high school to know about schools and staffing five years further education or work (school- from now when the data from the next to-work transition programs, tech- SASS administration are released. No one prep programs); would argue that an ongoing national survey monitoring conditions in the How schools are organized and schools should be redesigned to follow managed, including who makes every educational fad.Nevertheless, what kinds of decisions (site-based policies and priorities do change over management, decision-making time, and it is important for SASS to processes, choice, vouchers, remain relevant and timely. magnet schools, charter schools, networks).
Emerging Issues Implications for SASS
As we head toward the 21st century, some There is much we want to learn about of the major issues that education these reforms. However, it is important to researchers and policymakers are focusing pause and think about what SASS can and on include the following: cannot do well given its overall size and design. What students are taught (curriculum frameworks, course SASS should include only items that are content, graduation requirements); readily measured and that have a readily understandable definition. At the recent How students are taught NCES-sponsored conference on possible (instructional practices, use of future directions for NCES, Jennings and technology); Stark (1995, pp. 4-5) note this as a barrier to collecting information on school reform. How their progress is evaluated and They suggest establishing an advisory what are the results (tests, committee made up of individuals with portfolios); widely varying views on education reform so that they can help determine what to Who is teaching, how they prepare study and develop common definitions. for this responsibility, and what However, even if common definitions knowledge and skills they bring to could be agreed upon through such a bear (sources of teachers, minority process, we certainly cannot assume that teachers, content of teacher these definitions will mean the same thing education, professional to all the individuals answering questions development); about them. This is especially true for
215
221 reforms that are new and not yet clearly networks of schools with common goals, defined. such as the Coalition for Essential Schools. In a nationally representative sample, For example, the 1993-94 SASS asked however, you will not pick up enough schools if they offered a "tech-prep" schools to say anything useful about program. Tech prep was defined as schools that belong to this network or to "vocational-technical instruction in the last compare them with schools that belong to two years of high school designed to other networks. For now, if you want to prepare students for two years of study these networks, the best approach vocational instruction at the postsecondary would be to start with the list of level." With this definition, almost any participating schools.Similarly, one day school with a vocational program could we might want to track the movement of respond affirmatively to this question, and, teachers certified by the National Board in fact, 56 percent of public schools with for Professional Teaching Standards 12th grades reported that they had such a (Mandel, 1996, p. 3-30). At present, program (Henke et al., 1996). However, however, there are fewer than 1,000 such "tech prep" has now come to mean teachers. You might not sample any of something much more specific, with the them in a national survey, and certainly critical defining feature being some type of not enough to make any comparisons to formal articulation between secondary and other teachers. postsecondary programs. Consequently, not much has been learned from this Of the different types of reforms and question.This example also illustrates the changes occurring in schools and the dangers of trying to measure something teaching profession now, some are more too soon, before there is a common suitable for including in SASS than others. understanding of what it is.If a reworded It is relatively easy to ask who, what, or tech-prep question is included in the next how many. Much harder to answer, SASS, the percentage of schools with tech- however, are "how" questions. Thus, it is prep programs will probably appear to fairly easy to ask about professional decrease, which would not be an accurate development activities (how many depiction of what is actually going on. workshops of a certain duration a teacher NCES must guard against adding questions attended, for example, and in what subject on "hot topics" that do not yet have areas) or teacher education (measured in commonly understood or easily terms of courses, majors, minors, or communicated definitions. degrees).It is also relatively easy to measure how many students participated in SASS can only measure things that are certain programs (like tech prep), or how widespread. National surveys are not many plan to apply to college.It is also useful for providing information on feasible to ask about the existence of reforms that involve relatively small certain policies that have readily numbers of schools or teachers, as new understandable meanings (for example, structures and practices often do. For choice programs or vouchers). example, some schools are supporting each other in the reform process by establishing
216 Much harder to measure are organizational we can use that information to examine management and issues such as how implementation patterns by school and decisions get made. For these, national district characteristics. data collections such as SASS are less useful.In 1993-94, for example, teachers 2. More information is needed to and principals did not agree on how much describe what goes on at the classroom level. influence teachers have on certain types of In recent years, there has been increasing decisions. Although this is worth recognition that the quality of resources knowing, it does not give us a clear has to be measured at the classroom level. picture of what was going on in schools. Teacher quality and opportunities to learn, Because organizational processes are so not just district or school spending, which complex, I therefore disagree with Baker have traditionally been used as indicators (1996, p. 4) that SASS would be very of quality, are key to educational success, useful for studying these processes, and and therefore their distribution across believe that smaller studies using schools and states is of major national interviews would be more appropriate. A interest. The implication for SASS is that further complication is that the teachers in more information about what is going on SASS are not representative of the teachers in the classroom is needed. in their school. In fact, their perceptions about how their school works could differ Of the current SASS participants, teachers quite markedly from those of their are the best source of information on their colleagues. Samples for schools are too qualifications and on what is going on in small, for example, to permit HLM the classroom. They can be asked, for analyses despite the hierarchical structure example, what professional development of the surveys. activities they have participated in, if they are using a state curriculum framework, Implications for Survey Depth what assessment practices they are using, and how technology is used in the 1. NCES should take advantage of the classroom.Self-report data, even with fact that SASS, because of its linked-survey such straightforward items such as degrees design, is uniquely positioned to monitor the earned and dates, pose problems (Chaney, extent to which various types of reforms are 1994, pp. 8-12) but at the national level is present in schools and in the classroom. the best we can do. On a smaller scale, SASS is the best vehicle NCES has for self-report data can be combined with following how school reforms have been other types of data collection such as video implemented and for determining whether or case studies. or not state-initiated reforms have filtered down to the district, school, and Data Users and Uses classroom.States can develop wonderful curriculum frameworks, for example, but SASS has been targeted to a wide if teachers do not actually use them, we audience, including policymakers at all cannot expect to see their effects on levels, researchers, and the general public, student learning. SASS can ask teachers and some of their data needs differ. questions about instructional practices, and Researchers, while interested in national
217
2994. data, typically do not need the large Data Resources Center (NDRC). The data sample sizes required to support state-level would be even more accessible if NCES estimates. Education administrators, on were to create a Data Analysis System the other hand, want to know about (DAS) for SASS, such as that developed variation across states so that they can by Dennis Carroll for some of the other make decisions about funding, reform NCES databases. The DASs have allowed initiatives, and staffing (NCES, 1994, p. users with very limited computing capacity 1).Researchers are typically trying to and technical knowledge to create their establish links between educational own cross-tabulations. structures and practices and student outcomes, while administrators and The important point here is that rapid policymakers are more frequently advances in technology are opening interested in monitoring the exciting possibilities for innovations in implementation of proven or desired data collection, management, and policies and practices. distribution, and we are moving toward a closer connection between data producers From a survey depth perspective, the and data users.' For example, Statistics important question is whether the state- Canada recently held a symposium entitled and association-level data are used widely From Data to Information that explored enough to warrant continuing to produce topics such as the role of the customer in estimates at this level.It is difficult to "co-producing information," the know the extent to which state data are integration of data collection and analysis, used. NCES has published some state-by- advances in analytic techniques of state SASS data in the Statistical Profiles customers, and the privacy and and in SASS by State, produced jointly confidentiality issues associated with new with the Council of Chief State School customer/supplier partnerships (Scheuren, Officers and Horizon Research, as well as 1996). These trends suggest more, rather in the Digest of Education Statistics and than less, demand for and use of state and The Condition of Education, but it is other small area data in the future and a reasonably safe to say that these need for NCES to pay close attention to publications do not provide everything a how SASS data are being or might be state might need. used. In the past, users of NCES data have been fairly well known to NCES For state-level data to be really useful, staff, but as use becomes more dispersed, states need to have the capability to NCES may have to actively seek out the analyze SASS data themselves so they can users to find out who is using what data tailor comparisons to their specific needs. and for what purpose. The widespread availability of personal computers and appropriate software and This discussion of policy issues and data NCES's practice of providing large data users provides the context within which to sets on CD-ROM have made this proceed with the discussion of survey increasingly feasible. Moreover, a number depth. In the rest of this paper, I return to of states have obtained the data themselves the questions posed at the beginning: (1) or requested analyses from the National At what level should estimates be
218 provided? (2) From whom should data be cost of administering the survey. To collected? and (3) How much data can we provide estimates at this level, the collect from each respondent? In each 1993-94 SASS included approximately case, the issues are discussed first, 13,000 schools and 68,000 teachers. followed by specific recommendations. NCES could substantially reduce the sample size and therefore the cost of the survey if SASS provided only national At What Level Should Estimates estimates. Moreover, the reduction in Be Provided? sample size could free up resources to enhance the survey in other ways, such as Issues covering more types of school staff, conducting the survey more often, or The 1993-94 SASS sample design increasing the amount of information provides estimates at both the national and collected in each survey. The important state levels for public schools and question, then, is:Are state- and teachers, and at the national and affiliation affiliation-level data useful, or should the levels for private schools and teachers.In sample size be reduced and the resources addition, the sample design provides reallocated? Are there better ways to state/elementary and state/secondary spend the resources allocated to SASS than estimates; estimates of public and private having such a large sample? libraries and librarians at the national level; and national estimates of public and Response rates. Survey depth may also private school students by grade level and affect response rates, although the precise urbanicity. To improve the accuracy of impact, particularly of marginal changes, estimates for certain sampling units that is not easy to predict or measure.State- were relatively small in number but and affiliation-level estimates might important for policy reasons, SASS contribute to reduced response rates oversampled schools in which more than because the larger the sample, the greater 19.5 percent of the students were the probability that a given respondent will American Indian/Alaskan Native, included be surveyed more than once over time. all Bureau of Indian Affairs (BIA) Since respondents may view being asked to Schools, and oversampled bilingual/ESL, participate repeatedly as a significant Asian/Pacific Islander, and American burden they may be more likely to decline Indian/Alaskan Native teachers.' to participate a second or third time.
The level at which estimates are provided Providing state- and affiliation-level has implications for cost, response rates, estimates, requiring a larger sample, could and the precision of estimates.In the next jeopardize district response rates.In the section, the implications of each of these is 1991 LEA pretest, for example, response discussed in turn. rates were significantly lower for overlap compared with nonoverlap LEAs (84 Cost.Since the decision to provide state- percent compared with 95 percent), and affiliation-level estimates affects suggesting a reluctance to participate sample size, it has a direct impact on the repeatedly (NCES & U.S. Bureau of the
219 Census, in press, p. 39). This problem already lower than for public schools-83 could worsen over time as the amount of percent compared with 92 percent in overlap increases with repeated SASS 1993-94 (NCES & U.S. Bureau of the administrations, although the shift to a Census, in press, p. 4). Without five-year as opposed to a three-year association endorsements, the response rate interval between surveys will undoubtedly might be even lower. mitigate the problem. Reducing the number of questions asked of districts (by Precision of estimates.Finally, survey dropping them or shifting them to the depth has implications for the precision of school questionnaire) might also improve estimates. The optimum sample design for response rates for overlap schools. national estimates is different from that for state-by-state comparisons. For the best Repeated participation does not seem to state comparison, schools and teachers have had a negative effect on the school should be sampled in proportion to their response rate.In the 1993-94 SASS numbers in each stratum for national administration, the response rate for estimates, but equally among states.Since overlap and nonoverlap schools was about both types of comparisons have been the same among public schools (92 important, the sample has compromised on percent). Among private schools, overlap these goals (NCES, 1991, pp. 9-10). The schools actually had a higher response rate precision of the national estimates could be than nonoverlap schools--83 percent improved by abandoning state-level compared with 88 percent (NCES & U.S. estimates. Bureau of the Census, in press, pp. 39-40). The oversampling of certain types of teachers does not have cost implications Although there is no evidence to prove because the oversampling does not change this, it seems reasonable to hypothesize the total sample size.Larger samples of that providing affiliation-level estimates bilingual/ESL, Asian/Pacific Islander, and would affect the private school response American Indian/Alaskan Native teachers rate positively rather than negatively. are obtained by sorting the teacher lists so Private schools are not a "system" and that adequate numbers of these types of typically have less interest than public teachers can be selected. However, schools in making intrasector comparisons. oversampling does improve the precision The affiliation-level estimates allow private of the estimates for these groups. schools to define a more relevant comparison group than all private schools. Reconunendations For example, Montessori schools can compare themselves with other Montessori State- and affiliation-level estimates.I schools, and Catholic schools with other believe that a strong case can be made for Catholic schools or with other religious continuing to support state- and affiliation- schools. Without the affiliation-level level estimates given the purpose of SASS, estimates, private school associations the nature of emerging policy issues, and might not be willing to endorse the survey. the context in which school reform The response rate for private schools is initiatives are taking place.First, we need
220 226 state-by-state data to describe accurately through their own databases, SASS allows the education enterprise in the United comparisons with other states, which is States.Education is primarily a state and often hard to do when data do not come local responsibility, and the current trend from a common source. is away from federal involvement and uniformity at the national level, not toward Fourth, providing state-level data may it.Moreover, many important aspects of build support for NCES data collections schooling are typically controlled or with federal legislators because their heavily influenced by state policy, such as constituents will be happy. In a similar programs offered, curriculum, graduation vein, providing affiliation-level data will requirements, the number of days per year make the private school associations and hours per day that schools must be in happy, thus encouraging them to endorse session, pupil/teacher ratios, expenditures the survey, which, as suggested above, is per student, teacher certification likely to improve the private school and requirements, and salary schedules, to teacher response rates. name a few. Key features of the education system cannot be captured without state- Finally, it would be a shame to eliminate level data. state-level estimates just when the technologies being developed for data Second, the major current school reform collection, management, and dissemination efforts are being implemented at a are leading toward increased usefulness of subnational level, even those initiatives nationally collected data at subnational originating at the federal level. The levels. federal School-to-Work Opportunities Act, for example, has provided the states with The major argument against continuing to grants and given them much freedom to support state- and affiliation-level estimates design their own programs. In fact, there is cost. Cost savings could be a strong is much more talk in Washington about argument if there were better ways to use block grants than about new federal the funds. The major pressure right now programs. The reality is that most of the seems to be how to get more information major reform efforts are being initiated at from teachers.Eliminating state- and the state level.In the area of curriculum affiliation-level estimates would not do standards, for instance, states have moved much to accomplish this goal because the with varying strategies and at different chief difficulty is the response burden on speeds, with differences across states as individual teachers. noteworthy as the similarities (Cohen, 1995, pp. 11-12). To monitor reform, a Oversampled groups. Continuing to state perspective is needed.3 oversample schools with large American Indian/Alaskan Native student populations Third, state-level data provide and to oversample bilingual/ESL, policymakers with information useful for Asian/Pacific Islander, and American making state policy, such as scheduled Indian/Alaskan Native teachers also seems salaries. Although states can generate appropriate. The issues that prompted the some of this information themselves oversampling in the first place have not
221
227 disappeared. The only question would be whether other groups have greater priority, From Whom Should Data Be and this does not seem to be the case. Collected?
The American Indian/Alaskan Native Issues student population is relatively small (about 1 percent of the total student One of the major strengths of SASS is the population) and therefore would not be integrated survey design that links schools, well enough represented in a national teachers, districts (for public schools) and, survey of schools and staffing to permit as of 1993-94, libraries, librarians, and reliable generalizations about their students. Questionnaires are completed by characteristics. Yet there are policy districts, schools, principals, teachers, and concerns about the condition of education librarians. Two issues are important to for these students as well as some consider here. One is whether this is the systematic differences in the characteristics correct structure. Are we getting all the of these schools and staff compared with information we want, or should we include other schools and staff, and these bear other staff such as vice principals, monitoring (NCES, 1995a, p. 1). department heads, or counselors?
In the United States as a whole, 14 percent The other issue is whether we are asking of all children aged 5 to 17 spoke a the right questions of the right people. language other than English at home in There are two reasons why we might want 1990, and 5 percent had difficulty to change. One is to improve accuracy. speaking English. The percentage of For example, until the 1993-94 school-age children with difficulty administration of SASS, information on speaking English increased by 27 percent public school teacher benefits was between 1980 and 1990 (Smith et al., collected from the district, which was 1994, p. 130). Thus, it will continue to asked to identify from a list of benefits be important to monitor the adequacy of those that were offered to teachers. the supply of teachers to help these Beginning in 1993-94, questions on students. benefits were shifted to teachers, who were asked what benefits they received, Because Asian/Pacific Islanders and which was much more useful because it American Indian/Alaskan Native teachers provided more accurate information and made up such a small percentage of the allowed direct comparisons with benefits total teacher population (1.1 percent and received by principals. The second reason 0.7 percent, respectively, in 1993-94) we might want to change the source of (Henke et al., 1996), it would be difficult information is to reduce the response to study them without oversampling them. burden imposed on particular types of Obtaining accurate data on minority respondents. For example, if some teachers is important because the adequacy questions currently on the district of the supply of minority teachers and questionnaire could be shifted to the school their distribution among different types of questionnaire, the response burden for the schools remain crucial issues. districts could be reduced.
222 228 Recommendations Over the three administrations of SASS, NCES has switched the sources for some Since schools employ a wide variety of data (such as teachers' benefits) to types of staff, collecting information from improve quality.Overall, it appears that principals and teachers necessarily the right questions are now being asked of provides only a partial picture of school the right individuals to maximize accuracy. conditions. Other staff who could provide However, since the district questionnaire information on school conditions include takes the longest to complete, it is worth vice principals, department heads, considering whether the burden on the counselors, and school superintendents. districts could be reduced by shifting some However, the major problem with questions to the schools. Some district including these types of staff is that their policies would be known at the school roles vary widely from school to school, level, such as the number of days in the giving them quite different perspectives. school year or whether or not the district School superintendents would be a useful had a choice program (although schools source of information on both state and would not know how many students district policies, but probably no better participated).Collecting such data from than whoever fills out the district schools could lead to the problem of questionnaire.It might be interesting to reconciling data from multiple schools in learn about superintendents' backgrounds, the district.Also, it could irritate the education, and career paths, but this would school personnel filling in the not be a high priority given the ambitious questionnaire, who might wonder why the goals already outlined for SASS. questions were being asked of them rather than the district.Finally, some If extending the survey depth to collect information on district policies would only more information about what goes on in be available reliably from the districts, the classroom is considered desirable, the such as salary schedules, incentive pay question then arises as to who should policies, hiring criteria, and policies on provide the information, teachers or retraining. On balance, there does not students. The current Teacher Followup seem to be much prospect for shifting Survey (TFS) survey contains a much of the response burden away from considerable amount of information on the districts. The most burdensome teaching methods collected from teachers. questions are those that require looking up However, judgments on the usefulness of numbers, such as student enrollments, and this information will have to wait until the the district is the only source of this data on classroom practices are evaluated. information. Self-reported data on this topic may prove valuable, but could prove to have As a final point, it is worth keeping in significant limitations.This remains to be mind that the appropriate level to collect seen. Since the issue of whether to data may change in the future with include students in SASS is addressed in advances in data collection technology. another paper, it is not addressed here For example, information on enrollments (Kaufman, 1996). now collected from the schools may be
223
229 more easily obtainable from a district or To date, overall response rates have been even a state database. good, suggesting that the burden is notyet excessive.' However, theresponse rate to the initial mail survey (as opposedto the How Much Data Can We Collect? telephone follow-up) was considerably lower, suggesting that there may already Issues be too many questions forsome respondents. Although theresponse rate Once a survey has been developed, the has been increased by telephone follow-up, marginal cost of fielding an additional reinterviews have shown that theresponses question is relatively low.It is very obtained through the mailsurvey were tempting to keep adding questions, but the more reliable (Bushery, Royce, & response burden cannot be ignored. Kasprzyk, 1994, p. 7). Estimated times for completing the surveys (printed on the questionnaire) were as Recommendations follows: LEAs, two hours; public schools, one hour; private schools, two hours; and The major increase in response burden for principals, 30 minutes. The time teachers new data collection to address emerging took to complete the survey was not issues will fall on the teacher. For estimated, but the average time reported example, including all the information by public school teachers in 1993-94was asked on the TFS in the regular SASS 36 minutes. The overall burden on the teacher questionnaire would greatly principal may be considerably greater than increase the response burden for teachers. 30 minutes depending on how much of the A variety of options exist to address this, school questionnaire the principal has to including the following: complete personally. In a small school with no other administrators, the burden 1. Assume this is not a problem, and for responding to the school questionnaire add to the questionnaire. One could point and developing the teacher list probably to the high response rates attained in falls on the principal. previous administrations of SASS and argue that more questions could be added A twist on this issue is that the types of without reducing the response rate. This questions may be more relevant than the is an empirical question, at least fornow, length of time it takes to respond. and could be answered with a field test. Respondents may not object to completing a long questionnaire in which the questions 2. Try to maintain the amount of time are easy to answer, but may balk at a short that it takes to complete the questionnaire questionnaire that requires them to lookup while increasing the number of questions. numbers or that includes questions thatare Changes in technology may help make this puzzling or difficult to answer. Thus, possible. At some point in the not-too- NCES should pay close attention to the distant future, most teachers will have types of questions asked as well as how access to and be able to use computers. long they take to answer. Thus, it may be not only feasible but also very efficient to provide the questionnaires
224
f3 0 to teachers in electronic form, at least as information, the questions on majors and an option. A teacher might be able to minors may be enough. move considerably more quickly and easily through the questionnaire if he or she did 4. Do not ask all teachers all questions. not have to pay attention to skip patterns. This would require increasing the total Electronic data collection from teachers sample size to continue state- and and schools is already being investigated affiliation-level estimates for all data (Kasprzyk, 1994, p. 3).This could reduce items. The division of questions would the cost of data collection per teacher have to be done very carefully to ensure (making larger samples feasible) and that you did not split data among improve accuracy (by eliminating skip subsamples that you want to examine pattern errors, for example). To what together. extent it would reduce response time enough to permit asking a lot more 5. Cycle some questions. For example, questions is largely unknown, but bears on one survey you might ask about investigating. instructional practices and on the next perceptions of school problems or decision Also, as indicated above, it is not only the making. This means that some questions number of questions but also their would only be asked every ten years.In difficulty that affects response rate.If the reality this might not be a serious loss. questions are interesting to teachers and Comparisons of data from 1987, 1991, and easy to answer, they may not take much 1994 show considerable stability in many longer. areas. However, policymakers and the general public increasingly expect up-to- 3. Eliminate some of the current the-minute data (i.e., ten-year old data questions.It is very difficult to identify have little credibility). items that could be dropped from the teacher questionnaire, but a couple of 6. Use the TFS to collect data for which suggestions are made here. Among the national estimates are sufficient, to field test least useful seems to be the detail on what new data, or both. In 1994-95, the TFS is teachers were doing before they started being used to question teachers extensively teaching at that school (Questions 6-11). about their instructional practices. We do When they first taught and how many not know yet what the quality of these data years of total teaching experience they had will be, but if it seems worthwhile to could be sufficient.It may also be collect similar information again, the TFS unnecessary to include the detail on the may be an appropriate permanent home number of courses ever taken in various simply because there may be too many fields because this is a lot of work to questions to add to the regular teacher figure out and may not be very meaningful survey. The TFS is a good testing ground or accurate except for recent graduates. for new subject areas because the sample Moreover, this may be the type of is small yet nationally representative. question that causes teachers to give up. Since they could answer a lot of questions 7. Use the SASS sample as a framework in the time it takes to dig up all that for more limited studies. Metcalf (1995),
225 231 for example, has explored the feasibility of (3) Given the current focuson incorporating experimental designs into measuring quality at the classroom NCES data collection methodologies. If level, SASS needs to reachmore everything we are interested in cannot be extensively into the classroom. addressed in a national data collection, it This means obtaining detailed might still be possible to use SASS asa information on teacher training and sampling framework for additional, more professional development and the specialized studies. knowledge and skills teachers bring to bear (a direction in which the More information is needed to know which 1993-94 SASS has already moved), of these options (or combinations of and on instructional practices options) are most promising. In (beginning to be addressed through particular, we need to know how the TFS). successful the TFS data collection on instructional practices turns out to be as (4) State- and affiliation-level estimates well as the scope of the survey. should be continued, as should the current oversampling of schools with large American Indian/Alaskan Conclusion Native student populations and oversampling of bilingual/ESL, My major points can be summarized Asian/Pacific Islander, and briefly as follows: American Indian/Alaskan teachers.
(1) SASS has been an extremely (5) No additional types of respondents valuable survey, and its current should be added to SASS. structure of an integrated set of linked surveys is useful and (6) Because of the interest in focusing appropriate to meet the purposes of on the classroom, the increased the survey and to provide response burden will fall mainly on information on emerging policy teachers. Simply adding the TFS issues. questions to the next full teacher survey may not be realistic. A (2) SASS is the best mechanism NCES variety of options exist to address has for monitoring the this problem, but the results of the implementation and diffusion of TFS survey and decisions on many current school reform survey scope are needed before initiatives. recommendations can be made.
226 References Jennings, J. F., & Stark, D. (1996). Tracking educational reform: What type Baker, D. P. (1996). Towards an of national data should be collected organizational database on America's through 2010? In G. Hoachlander (Ed.), schools: A proposal for the future of From data to information: New directions SASS, with comments on school reform, for the National Center for Education governance, and finance. The Schools and Statistics (NCES 96-901). Washington, Staffing Survey: Recommendations for the DC: U.S. Department of Education, future (NCES 97-587). Washington, DC: National Center for Education Statistics. U.S. Department of Education, National Center for Education Statistics. Kasprzyk, D. (1994). The Schools and Staffing Survey: Research issues.Schools Bushery, J. M., Royce, D., & Kasprzyk, and Staffing Survey: Papers presented at D. (1994). The Schools and Staffing meetings of the American Statistical Survey: How reinterview measures data Association (NCES 94-01). Washington, quality. Schools and Staffing Survey: DC: U.S. Department of Education, Papers presented at meetings of the National Center for Education Statistics. American Statistical Association (NCES 94-01). Washington, DC: U.S. Kaufman, P.(1996).Student-level data: Department of Education, National Center If, when, and where. The Schools and for Education Statistics. Staffing Survey: Recommendations for the future (NCES 97-587). Washington, DC: Chaney, B.(1994). The accuracy of U.S. Department of Education, National teachers' self reports on their Center for Education Statistics. postsecondary education (NCES 94-04). Washington, DC: U.S. Department of Ligon, G. D. (1996). New developments Education, National Center for Education in technology: Implications for collecting, Statistics. storing, retrieving, and disseminating national data for education. In G. Cohen, D. K. (1995, December). What Hoachlander (Ed.), From data to is the system in systemic reform? information: New directions for the Educational Researcher, 24(9), 11-12. National Center for Education Statistics (NCES 96-901). Washington, DC: U.S. Henke, R. R., & Choy, S.(1996). Department of Education, National Center Schools and staffing in the United States: for Education Statistics. A statistical profile, 1993-94 (NCES 96- 124). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
227
233 Mandel, D. R.(1996). Teacher National Center for Education Statistics. education, training, and staff development: (1995b). Programs and plans of the Implications for national surveys. In G. National Center for Education Statistics Hoachlander (Ed.), From data to (NCES 95-133). Washington, DC: U.S. information: New directions for the Department of Education, Office of National Center for Education Statistics Educational Research and Improvement. (NCES 96-901). Washington, DC: U.S. Department of Education, National Center National Center for Education Statistics, for Education Statistics. U.S. Department of Education, & U.S. Bureau of the Census.(in press). Metcalf, C. E.(1996). Incorporating 1993-94 Schools and Staffing Survey: experimental designs into new NCES data Sample design and estimation (NCES 96- collection methodologies. In G. 089). Washington, DC: U.S. Department Hoachlander (Ed.), From data to of Education, Office of Educational information: New directions for the Research and Improvement. National Center for Education Statistics (NCES 96-901). Washington, DC: U.S. Scheuren, F. (1995). Administrative Department of Education, National Center record opportunities in education survey for Education Statistics. research. In G. Hoachlander (Ed.), From data to information: New directions for National Center for Education Statistics. the National Center for Education Statistics (1991).1988 Schools and Staffing Survey (NCES 96-901). Washington, DC: U.S. sample design and estimation (NCES 91- Department of Education, National Center 127). Washington, DC: U.S. Department for Education Statistics. of Education, Office of Educational Research and Improvement. Scheuren, F. (1996, January). From data to information. Amstat News, 227, p. 13. National Center for Education Statistics. (1994). SASS by state (NCES 94-343). Smith, T. M., Rogers, G. T., Alsalam, Washington, DC: U.S. Department of N., Perie, M., Mahoney, R. P., & Education, Office of Educational Research Martin, V. (1994). The condition of and Improvement. education 1994 (NCES 94-149). Washington, DC: U.S. Department of National Center for Education Statistics. Education, National Center for Education (1995a).Characteristics of American Statistics. Indian and Alaskan Native education (NCES 95-735). Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.
228
224 1. Two papers presented at the conference on Future NCES Data Collection: Some Possible Directions address this issue.See Scheuren (1995) and Ligon (1995).
2. This is somewhat of an oversimplification of the sample design. For more detail, see NCES and U.S. Department of the Census (in press).
3. Jennings and Stark (1995) recommend that NCES consider studying education reform on a state-by-state basis and issue annual reports on state activities.
4. In the public sector the weighted response rates in 1993-94 were 93.9 percent for districts, 96.6 for principals, 92.3 percent for schools, and 88.2 percent for teachers.In the private sector they were 87.6 percent for principals, 83.2 percent for schools, and 80.2 percent for teachers (NCES and U.S. Bureau of the Census, in press, p. 4).
229 235 REFLECTIONS ON TILE PAPERS PREPARED FOR THE SCHOOLS AND STAFFING SURVEY SEMINAR SERIES
John Howard Burkett, Wilmington, NC
Education: n. That which discloses to the wise and disguises from the foolish their lack of understanding. Ambrose Bierce, The Devil's Dictionary
Preface are presented "baldly" as signs of the need for greater clarity. The following Reflections are offered as a layman's contribution to the ongoing The overall aim, however, is not to argue discussion and planning of a "new" specific points but to try to explore a Schools and Staffing Survey (SASS) by the "global" perspective on what NCES might National Center for Education Statistics do with SASS in light of the extensive (NCES) beginning in 1998-99. They are technical work done on SASS in the past based on a review of commissioned papers and in these commissioned papers.In this from the perspective of an (allegedly) aspect, the author will probe some educated citizen, familiar to a degree with assumptions and proposals presented in the the work of SASS and NCES but papers. Such probing is not intended to be decidedly not a technical expert in definitive but "framing," in the sense of statistics or survey research.It is hoped identifying major points of agreement and that these Reflections will stimulate disagreement, and of suggesting lines of discussion on the future content and design further inquiry and possible resolution of of SASS. contentious issues. Above all, the Reflections are a starting point, neither In what follows some specific criticism of conclusions nor definitive points made in the papers will be ventured. recommendations. In some (if not most) cases, these criticisms may be misplaced, betraying the Finally, the author apologizes if any of the author's nonexpert standing.If such there writers of the commissioned papers be, these mistakes will indicate where a believes his or her work has been slighted "layman" has trouble with understanding or maligned in this Reflections. The just what the experts are trying to say. response is simply that the task here is But the ultimate "test" of the work of a general.It should be clear that a great public statistical agency is its ability to deal has been learned from these papers communicate clearly to the public and that such value as these reflections information relevant to the issues of may have is due in large part to that concern, in this case how the nation's learning (or misunderstanding). education is faring. Thus, these criticisms
231
roLW9 fqs SASS and the Public noting "I have always wantedto know how my children's school was doing andnow I As is well known, and universally am beginning to find out." While the acknowledged by the authors of the usual issues of school politics dominatethe commissioned papers, the data needs in upcoming school board elections in New education have expanded almost Hanover County, I have been struck by the exponentially in the last decade anda half. fact that in public forumssome citizens While there has been a recognition of the have asked questions stemming from the need for statistics (literally "state release of the "Report Card." information") since the founding of the nation, this need has been largely confined The near identification of improvement in to small groups of people. And, as the education with enhanced economic well early resistance to a census (and the latest being (if not survival) is now complaints about it) indicates, the public commonplace. The entire "Ideas" section generally has shown less enthusiasm for of the Sunday, April 21, 1996, statistical enterprises.In spite of Greensboro, NC News & Recordwas Americans' love of information (witness devoted to education, under the general the proliferation throughout our history of heading "Will future 'State of the South' atlases, encyclopedia, etc.) development of rise? Our fate awaits school bells." While thorough statistical surveys has always some of us may view such claims as far proved a difficult sale. too simplistic, the plain fact is that in some fundamental way the claims have There has been, I believe, a sea-change in taken hold and the public is demanding attitudes toward the importance of statistics that schools do a better job. And in all in education. Late this winter, the New this, the value of good statistical Hanover County Schools in North Carolina information is being recognized. issued its first "Report Card"--a 14-page newspaper section in the Wilmington Herein, of course, lie grave difficulties. Morning Star prepared by the school The public (including you and me) is district, giving detailed information about fickle. Worse, in education, the most the school district and about each school outstanding feature is the rise and fall of within the district.Comparisons were fads. We all want the solution, and, made of each school to the district especially if it concerns our own children, averages for each type of school "We want our Maypo and we want it (elementary, middle, and high) as wellas now." Statisticians and serious social comparisons of the district to both state science researchers are painfullyaware of and national data in some areas. the dangers of responding merely to the latest fad.Instant polling and immediate Though obviously limited in many ways, reporting of political trends clearly the "Report Card" was extensive and, most demonstrate how even well-designed importantly, has struck a responsive chord instruments yield misleading information. in the community. I was particularly And given the rapidity with which taken by a letter to the editor, praising the education fads change, we know that paper for publishing the publication and
232 designing surveys to provide that Maypo sense of that term), who must respond now is worse than useless. directly to the public needs and desires. And it is thus to that audience we must But the fact remains that there is a public turn in asking what SASS can be as an need for timely and reliable and valid data important part of the public debate about on education. The early success of SASS education. in responding to a concern ("teacher shortages") demonstrates that a survey But, again, I want to insist that keeping in program of real quality can arise from an mind what the public wants and needs to immediately pressing need and advance the know are central to the enterprise. That public understanding of the issue.Such danger in focussing on policymakers is that work can advance that understanding and we fail to remember that they are open new avenues of inquiry. especially subject to the whims of public opinion and pressure groups. Without a My first point, then, is that there is a touchstone, we risk merely pleasing the pressing public need for more information "masters" without addressing the public on just how schools are faring. Though needs. seemingly trite, this point has, I believe, an important implication for thinking about SASS: attention must be paid to what the Policymakers as Audience public wants and needs to know. While there appears to be general Well, of course, you may reply, but so agreement that policymakers are the what? Put as strongly as possible, in all primary audience for SASS, it is not clear our deliberations, the ultimate touchstone just what that entails.Policymakers no is whether the information will make sense less than the public at large primarily want to the public! Put another way, we need to do something--now. As NCES's constantly to ask, "Would I, as a parent experience with the National Educational and citizen, gain information from SASS Goals Panel shows (at least through 1994), that will make a difference in how I policymakers are impatient with the approach my children's education and that limitations of statistical information--both of all the children in my community?" in scope and timeliness.In many instances, NCES was able to persuade the A noble thought, but how can we seriously Panel that these limitations be accepted. address the public at large? Notice that I That experience (at least for me) have insisted that we take making sense to demonstrates the value of carefully the public as a touchstone for work in designed surveys and studies and the SASS. By that I do not mean that SASS ability of policymakers to grasp that value. should become the USA TODAY of the education world. For while public What is needed, I suggest, is the kind of concerns are the basis for the work, it is to approach set forth in general terms in the public policymakers that the work must Professor Boe's paper (pp. 185-212) for be primarily addressed.It is the this seminar. In it, you will recall, he policymaker, the politician (in the best argues for three "guidelines" for selecting
233
238 aspects of the educational enterprise to of schooling must be explicitly and include in SASS data collections.I carefully set forth.I do not argue for or paraphrase from Boe's paper (p. 190): against the particulars he sets forth but for the general approach and especially for the (1) Select topics that have been subject need to keeping such a classification at the to major policy action in the recent forefront of thinking about SASS. past that focus on "fundamental aspects of schooling." Which Policymakers? (2) Select topics that are subjects of recent analyses and debates but not Boe argues that policymakers at all levels of major policy action. should be the primary audience. In many ways there is little to disagree with in his (3) Select topics of major public claim. However, as Choy notes, concern. "Education is primarily a state and local responsibility, and the current trend is These guidelines and Boe's discussion of away from federal involvement and them capture the core of what I think uniformity at the national level, not should be SASS's approach. His argument towards it" (p. 221). The situation is is simple and straight-forward: one cannot further complicated by the fact that, do everything but one can take what are among the states, there is great variance in basic public concerns, as reflected in the level of state versus local control in policy actions and debates, and in general education.' The fact remains, I believe, public discussions. As Boe notes (p. 192), that it is primarily state and local what needs to be focussed on are those policymakers that should be the focus. aspects amenable to policy intervention.' If this be conceded, the suggestion in Boe also reminds us of the importance of Blank's paper needs to be taken seriously, distinguishing between monitoring what is namely, that NCES should work closely happening in education and specifically with state education information personnel. advocating any particular policy action He draws a useful parallel to the way in (pp. 187-189). While a well-rehearsed which the Common Core of Data was slogan of the work of NCES, this improved with the Cooperative Statistics distinction is often lost on the audience and program. In any event, it seems to me even occasionally within the statistical that the value of SASS will ultimately be community.' Frustrating as it may be, the realized only if it addresses education at desire to satisfy specific policymakers' the state and local level. The federal concerns must be resisted.Professor Boe needs must be secondary except in so far offers good counsel on this issue. as a federal role in providing some level of uniform national data so that states can The upshot here is that something like make meaningful comparisons. Boe's classification of fundamental aspects
234 239 What about Educators? which reforms are discussed--a point that parallels the one Choy makes as noted') If SASS is truly successful in providing a rich picture of schools and staffing, surely this information would be of value to those Researchers and SASS who formulate and implement specific educational policies. Blank urges that data In the argument above, I have stressed the must be of more use than just for view that SASS is primarily for education monitoring the enterprise (p. 166). His policymakers, and particularly for state argument is mainly based on the practical and local authorities in education. But, it considerations of retaining teacher/ should be noted, SASS (and indeed all of administrator cooperation in data NCES's work) has always been conceived collection.If we want the best data we as providing data for educational research will need that cooperation, so his point is as well as for policymaking. well taken. In some ways there is a false dichotomy There is, however, a danger in here. Data useful for policy debate and concentrating on this point in the general formulation are not devoid of value for design of SASS. That danger parallels the research purposes. And clearly, research one mentioned earlier about following can inform the means for collecting and mere fads in the public concerns. While analyzing data for policy uses. More appreciating the practical argument of importantly, I think it important to securing participation (and surely his distinguish between designing a survey for suggestions about communications with informing policymaking and designing one participants are sound), I caution against for research purposes. Large scale taking this point as a central focus. surveys are not essential for research, Instead, we need to think about the though their existence provide a context usefulness of SASS for teachers and school for it.For example, as Stodolsky notes, administrators, but more in terms of what large-scale surveys have been valuable but such a survey would provide for research finally are no substitute for observational studies that can be of use to teachers. studies and other research activities in examining curriculum and instruction (p. (I do want to note that if teachers are seen 5). On the other side, the importance of as a more direct audience than I have using research to design an effective argued, Stodolsky's paper raises some survey is unmistakable. important issues about what should be involved in such a survey. Of special To consider ways in a large-scale survey import is her account of needing to know that research can be conducted is of course about what teachers really understand sensible. But not at the cost of limiting reform to be. Perhaps for policymakers survey data to the research hypotheses. including something about teachers' Thus, while Stodolsky argues that to understanding of reform initiatives would understand the new developments in be of value. Note, too, her point about teaching ("the constructivist approach") the diverse understanding of the terms in SASS should include elements to examine
235 240 this phenomenon (or link with other astray in creating too many nonacademic studies), to do so would, I am concerned, roles for schools to perform. Robert muddle the role of SASS as a general Spillane has often argued that if you could monitor. Something like Boe's distinction get all the other roles out, the school between policy .monitoring and policy would do much better in their academic advocacy needs to be made here. mission. In any event, any serious Sometime we just must say no. redesign of SASS must address ways to explore ways to relate the conditions in In several of the papers mention was made schools and staffing with academic about how limited has been the research achievement. work based just on SASS.I am not certain that this fact constitutes a major What parents and public want to know is criticism.I acknowledge there are how well are the schools doing in technical issues about how readily the data academic achievement. And, as all our in SASS can be linked to other data, etc., seminar writers seem to agree, a well- but I do not see these as of primary designed SASS can be an important concern in the sense that these needs component of the database to answer this should shape major directions of SASS.' question.Further, with such information If the earlier discussion has any merit, the about the conditions of schools and value of SASS for policy monitoring far staffing, richly elaborated, our knowledge outweighs any direct research limitations. of "what works" in teaching and learning And, as suggested, we may not have a real could be greatly enhanced. either/or here. Kaufmann has explored ways that SASS could be shaped and/or related to NAEP, The Big Issue NELS, and other studies of student outcomes. Detailed discussion of his The above discussion has left one major proposals is beyond my competence. issue hanging and gives an air of McLaughlin's Comments on Kaufmann' "abstractness" to the discussion. While deserve our attention, and I eagerly await there are many aspects of schooling that that discussion. are of interest to the public, the heart of the matter is whether students are learning One issue raised by McLaughlin needs what supposedly they are being taught. emphasis: the linkages that are being That is, student achievement, the outcome developed through NAEP and developing of prime importance in education, has not state assessment systems (McLaughlin,' been discussed. pp. 4-5).Consistent with the position Blank takes, McLaughlin's proposal for To lay the cards on the table, though one linking state-produced school-level can talk about the many outcomes that achievement data with SASS data on those schooling may have, it is for the purpose schools is promising. Clearly, the states of formal academic instruction that schools are now forging ahead with reforms and have been established. Some of us are data collections including achievement prepared to argue that society has gone data. SASS can offer information that
236
24,E provide the states with contextual Fulton and Gilford each argue for information that is genuinely useful to some detailed considerations of states. The key here, as McLaughlin distinct aspects of schooling- - stresses, is that the focus must be on technology and professional providing "school dynamics data to states development activities in school, for use in research, by states [emphasis respectively. Without discussing added] ..." (McLaughlin,' p. 5). the details of their presentations, let me suggest that these topics A key technical question that must be represent just the sort of items settled is whether McLaughlin is right that about which we need the prior aggregate and cross-sectional student data thought about how fundamental (in is sufficient for addressing important Boe's sense) these activities are to SASS-related questions concerning student an understanding of what is achievement. If he is, the approach he happening in schools. Is technology proposes is worthy of pursuit. just a fad? More importantly, without linkage of SASS to student One issue to keep in mind, especially achievement data, information when addressing an educated citizen who about technology is vacuous. wants to see schools improve: when most (Without a doubt nearly everyone parents express concerns, they are now looks to technology as a kind fundamentally focussed on how his or her of deus ex machina in education, child or children are doing. Nothing in but there are a few of us Luddites SASS can answer that concern directly, of left.)Professional development course. But SASS, in the context of activities strike closer to home in developing state assessment and reporting my view, and Gilford's suggestions programs, can provide useful information. may prove useful in certain aspects Forging links between SASS and state of the teacher survey. agencies, as both McLaughlin and Blank suggest, not only provides a practical Ross's paper demonstrates my reason for cooperation with data collection concern about a too narrow focus but makes SASS an integral part of a truly on particular federal programs and national data system for education. reform that, as his own presentation makes plain, is understood in so many different ways. To be sure, Some Passing Thoughts there have been federal programs in support of magnet schools, but I do I do not want to slight papers not not see much value in any detailed mentioned in the above discussion, but my survey of this phenomenon in purpose has been to focus on what I take SASS. One place that some of the to be the heart of the matter. Let me concerns herein expressed might be make a few comments about some of the explored is in the resource aspects other papers, noting some points of of the survey, especially if some of interest and/or concern. what Chambers suggests prove of use.
237
242 Chambers' paper reminds us that Final Thoughts merely gathering fiscal data may offer us very little real information. In the above I have argued fora vision of Dollar allocations are in themselves SASS as an integral part ofa developing sterile.His suggestion to tie those national data system in education by being dollars to real activities is on the a tool for use by state and local mark. educational agencies.'I have also argued that the primary focus should be in Zheng argues for the continued use providing school and staffing data, linked of questions about teacher and to student/school achievement data, most principal perceptions of school useful to state and local policymakers.I conditions. But Chambers reminds also have insisted that such a focus dictates us that objective measures of restraint in thinking about how much ofa events, activities, and behaviors are research instrument SASS should be. far better sources of information. There are many things researchers would After all, as Chambers notes, "If like to know, that would be important and schools do not maintain reporting interesting to know, but that are not mechanism on certain student feasible in a survey designed for the behaviors or activities, then one purposes I have suggested. That the SASS might question whether or not the design be informed by the best research activity is really a serious problem" available, and that SASS provide data (p. 182). useful to researchers, is without doubt.It should not be, in itself, a research Baker's paper deserves much more instrument. attention than I have given it.I have not yet sorted out all the Nor should SASS be seen as a federal aspects of his claim a new view of instrument to monitor and evaluate federal organizations would provide. education programs. SASS can provide, Intuitively, the idea of looking at as it has, data on special populations that organizational processes rather than may be useful for federal and state structures seems right.But, with programs, but the focus should always be Choy, I worry that the complexity on what is happening fundamentally in the of these processes would render schools and how that is related to student any general survey data so general achievement. In this light it is useful to as to be useless. Baker's view is keep in mind Boe's point about identifying just the sort of thing I mean as a genuinely fundamental aspects of schooling research contribution to the SASS and keeping these at the center of design. Whether he is right, he attention. asks us to consider a different theory of organizations and how it Without such a strict focus, SASS may might provide insight into what all become the perennial "Christmas tree," acknowledge as the complex resembling more a congressional interactions that formal education appropriations bill than a serious survey represent. instrument.
238
24.3 On Learning
Learning is a painful experience because we learn from mistakes, which are painful, and because in learning we give up something of ourselves, some old belief or action.Thinking is unpleasant not because it demands effort and concentration (so does love) but because our thoughts come trailing wisps of anxiety -we might learn something.It is not surprising that alternatives to taking thought are attractive, especially if they present themselves not as thoughtlessness but as conclusions of profound and proven reflections.The fallacy embedded in this pattern may be called sloganizing.
Abraham Kaplan In Pursuit of Wisdom, p. 146
239 244 1. Susan Choy has an interesting discussion of "emerging issues" (pp. 215-217) consistent with Boe's guidelines. She rightly reminds us that in addressing such issues real problems of common understanding of terms are present. See, for example, her point about what 'tech prep' means.
2. There is a special aspect of this issue having to do with the role of researchers as users of NCES data that will be discussed later in this paper.
3. In its initial reports the National Education Goals Panel used statements about how much fundingcomes from federal, state, and local sources for education, citing the average figures. As NCES pointed out, these average figures masked a great variation among the states in the level of support from local funds. Deep Throat said, "Follow the money." Always sage advice!
4. Well, to be fair, there is one critical issue, discussed below--linking to student outcomes.
5. Discussion of Kaufmann's proposals is contained in McLaughlin, D., Comments on Linking Student Data to SASS: Why, When, and How, (unpublished paper), American Institutes for Research, Palo Alto, CA.
6. Ibid.
7. Ibid.
8. I have not spoken at all of private education.I do not mean by this omission to imply that there is no role for a private sector SASS. Indeed, I would argue that the private system is part of an overall public educational system. States charter private schools, and in many states "charter schools" are becoming an explicit part of public education.I predict that in the years ahead much more of this sort of schooling will emerge. And this will be the result of public policy decisions. Thus, SASS will need to be open to this sort of development as well as continuing its more narrowly defined private school dimension.
240
245 CONTRIBUTORS
SUSAN S. STODOLSKY has been an education professor at the University of Chicago for more than 25 years and currently serves as chair of the Measurement, Evaluation, and Statistical Analysis Program within the University's Department of Education. She has written more than 50 books, chapters, articles, and papers on education issues, including compensatory education, learning patterns, instructional grouping, instructional practice, and educational evaluations.
DAVID P. BAKER is Associate Professor of Sociology at the Catholic University of America and a Senior Research Scientist (part-time) at the American Institutes for Research. From 1992 to 1994, he was the AERA Senior Fellow at the National Center for Education Statistics; his primary research interests include school organization, school achievement and opportunity to learn, and private education in the United States.
KATHLEEN FULTON is currently the Associate Director of the Center for Technology and Education at the University of Maryland at College Park. Previously, she served as the project director and senior analyst with the Congressional Office of Technology Assessment and as a policy analyst and program analyst for the U.S. Department of Education.
PHILLIP KAUFMAN is Senior Research Associate at MPR Associates and has directed several major studies using data from national databases such as the National Education Longitudinal Study: 1988, the Current Population Survey, and the Schools and Staffing Survey. Prior to joining MPR, he served as a statistician at the National Center for Education Statistics, working on the Condition of Education and with Elementary/Secondary Outcomes.
HENRY Y. ZHENG is currently a Ph.D. candidate in Public Policy and Management at Ohio State University. He recently served as an AERA Research Fellow at NCES. He has worked on data analysis and planning for the Ohio Department of Youth Services and the National Gang Research Center, focusing on changes in Ohio's juvenile offense patterns and institutional intake trends.
DOROTHY M. GILFORD is a mathematical statistician with a history of providing leadership in mathematics and educational statistics at federal, national, and international levels for more than 40 years. She served as Director of the Mathematical Services Division of the Office of Naval Research and as Assistant Commissioner for Education Statistics and Director of the National Center for Education Statistics. She also held positions within the National Research Council/National Academy of Sciences, including Director of Human Resources Studies and Director of the Board on International Comparative Studies in Education.
J. MICHAEL ROSS serves as a statistician in the Education Surveys Branch of the National Center for Education Statistics and had previously served as a statistician in the assistant attorney general's office of the civil rights division at the U.S. Department of Justice. Earlier in his career he spent ten years as a sociology professor at Boston University.
241
246 ROLF K. BLANK has served as the Director of Education Indicators in the State Education Assessment Center of the Council of Chief State School Officers since 1986. Priorto joining CCSSO, he was a research associate in the Division of Social Sciences and Educationat the National Academy of Sciences.Dr. Blank has a special interest and expertise in state curriculum frameworks, mathematics and science education, and magnet schools.
JAY G. CHAMBERS is currently Senior Research Fellow and the Director of the Education and Public Sector Finance Group of the John C. Flanagan Research Center at the American Institutes for Research. He has directed numerous projects for the U.S. Department of Education, thestates of California, Massachusetts, and Kentucky, and various foundations on a widerange of issues, including Title I resources, teacher supply and demand, special education, childcare, and comprehensive services.
ERLING E. BOE is Professor of Education in the Graduate School of Education and Co-Director of the Center for Research and Evaluation in Social Policy at the University of Pennsylvania. He has taught at University of Pennsylvania for the past 30 years and served as a visiting expert to the National Center for Education Statistics in 1989, working on numerous projects in teacher supply, turnover, and shortages.
SUSAN P. CHOY is Vice President of MPR Associates. She was project director fora task order contract that provided data analysis and programming support to the Elementary/Secondary Education Statistics Division of NCES. Many tasks completed under this contract used data from the Schools and Staffing Survey. Over the past ten years, Dr. Choy has conducted numerous studies for NCES on both elementary/secondary and postsecondary education topics using many NCES surveys.
JOHN HOWARD BURKETT served as special assistant to the assistant secretary at the U.S. Department of Education, Office of Educational Research and Improvement for four years. Prior to comingto the federal government, he spent more than 20 years in higher education, firstas a professor of philosophy and then in several administrative posts, including the Assistant to the President at Franklin College, St. John's College, and Boston University.
242
247 United States Department of Education Postage and Fees Paid Washington, DC 20208-5651 U.S. Department of Education Permit No. G-17 Official Business Penalty for Private Use, S300 Standard Mail (B)
BEST COPY AVAILABLE 248 U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement (OERI) Educational Resources Information Center (ERIC) RIC
NOTICE
REPRODUCTION BASIS
This document is covered by a signed "Reproduction Release (Blanket)" form (on file within the ERIC system), encompassing all or classes of documents from its source organization and, therefore, does not require a "Specific Document" Release form.
This document is Federally-funded, or carries its own permission to reproduce, or is otherwise in the public domain and, therefore, may be reproduced by ERIC without a signed Reproduction Release form (either "Specific Document" or "Blanket").
(9/92)