DOCUMENT RESUME
ED 442 704 SO 031 820
TITLE WEEA Digest on Education Assessment. INSTITUTION Educational Development Center, Inc., Newton, MA. WEEA Equity Resource Center. SPONS AGENCY Department of Education, Washington, DC. PUB DATE 2000-03-00 NOTE 13p. CONTRACT ED-98-00-0008 AVAILABLE FROM Women's Educational Equity Act (WEEA) Resource Center, Education Development Center, Inc., 55 Chapel Street, D-0300, Newton, MA 02458; Tel: 800-225-3088; Web site: (http://www.edc.org/WomensEquity). PUB TYPE Collected Works General (020) EDRS PRICE MF01/PC01 Plus Postage. DESCRIPTORS Accountability; *Cognitive Style; Disabilities; Elementary Secondary Education; *Evaluation Methods; *Gender Issues; *School Culture; *Student Evaluation; Student Experience
ABSTRACT The March 2000 WEEA Digest contains two articles. The first, "Honoring Ways of Knowing"(A. Lin Goodwin) mentions that notions of "educational excellence" or "education for all" are too seldom actualized because such conversations often sidestep the reality of inequitable educational practices. Voluminous data exist that articulate how schools help to structure inequality. This digest begins with a brief summary of school experiences of children of color and girls in an effort to make known the ways that schools structure inequality and educators'(and society's) unrelenting low and limiting expectations for these students. The article discusses how authentic assessment can precipitate a shift away from knowledge as discrete and intelligence as static, and foster teachers' deeper understanding of children of color and girls' abilities, gifts, and ways of knowing. It argues that authentic assessment can result in transformative teaching that honors children's diversities and multiple ways of knowing and learning to nurture "all" their talents. The digest includes lists of WEEA resources on assessment; additional resources; and resource organizations for assessment. The second article entitled "Academic Accountability for Students with Disabilities and LEP" (Tim Boals) reports on how Wisconsin has developed an alternative assessment framework for special students.(BT)
Reproductions supplied by EDRS are the best that can be made from the original document. sa Oil N .411
6 EquityE '4 Resource Center 044E14.44-1400b4044441.614 Honoring Ways of Knowing By A. Lin Goodwin, Teachers College, ColumbiaUniversity
Notions of "educational excellence" or "edu- understanding of children of color and girls' abili- cation for all" are too seldom actualized be- ties, gifts, and ways of knowing. In the article, I cause such conversations often sidestep the argue that authentic assessment can result in blatant reality of inequitable educational prac- transformative teaching that honors children's tices. Voluminous data exist that articulate how diversities and multiple ways of knowing and schools help to structure inequality, ensuring learning, and nurtures all their talents. U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement that all students, particularly those outside the EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) power culturestudents of color' and girls Variability in School Experiences ] This document has been reproduced as are afforded limited or uneven access to learn- received from the person or organization This section begins with a basic assumption: originating it. ing opportunities.2 Thus, despite the many that each of us is multiply identified. Any dis- Minor changes have been made to positive changes that have occurred to support improve reproduction quality. cussion that attempts to separate race fromgen- racial, gender, and class equality, middle-class der, from class, from language, from heritage, Points of view or opinions stated in this white boys continue to outstrip girls and chil- document do not necessarily represent and so on, is inherently faulty;we each are official OERI position or policy. dren of color in terms of achievement,access shaped by our cultures, see the world through to resources, vocational choice, our experiences, and are culturally complex. Inside: and life options. While I do acknowledge that children's experi- WEEA Resources 3 Some theories about inter- ences in schools cannot be simplistically framed Additional Resources rupting this persistent trend have by any individual cultural characteristic,my emphasized the disjuncture be- review nonetheless relies upon a body of lit- Organizations 9 tween the dominant paradigm erature that sometimes accentuates one aspect Academic Accountability that frames schooling and the of identity over or to the exclusion of others. for Students with multiplicities demonstrated by di- My discussion is not meant to be exhaustive, Disabilities and LEP 12 verse learners.' Schools conform but rather is designed to highlightsome of the to and perpetuate narrow concep- different ways in which children of color and tions and measurements of intelligence, know- girls encounter school. In it, I drawon two in- ing, and success, views that invariably find chil- formative reviews5 of the differential schoolex- C:1 dren of color and girls wanting!' One cannot periences in relation to gender and racecon- help but wonder what would happenwere we ducted by Carter and Goodwin (1994) and N to change both the way we teach and how we Grossman and Grossman (1994). 00 assess what children know. What might this Teaching and learning are reciprocal and mean for the educational attainment of those cyclical actions that occur primarily through the children who are served least well by schools? interactions that students and teachers have This article begins with a briefsummary of with one another. It is often the quality of these C4f) the school experiences of children of color and interactions that determines the quality of one's O girls in an effort to bring to the surface theways educational experience. Studies have shown in which schools structure inequality and edu- that a qualitative difference exists in the inter- cators' (and society's) unrelenting low and lim- actions teachers have with children of color and iting expectations for these students. It then dis- with girls. Girls are less likely to be praised, cusses how authentic assessment can precipitate called upon, or given positive feedback than a shift away from knowledge as discrete and in- their male classmates. Teachersare more likely telligence as static, and foster teachers' deeper 2 BEST COPY AVAILABLE Continued p. 2, "Ways of Knowing"
March 2000 Women's Educational Equity Act (WEEA) Resource Center at EDC, 55 Chapel Street,Newton, MA 02458800-225-3088 Education Assessment 2
Ways of Knowing ...continued to afford boys additional response time and in- cally capable than boys, yet view girls of color structional assistance, to integrate their ideas into as less able than white girls, and boys of color classroom discussions, and to offer them encour- as less able than girls of color. Again, the im- agement. African American and Latino children, plicit hierarchy mentioned earlier becomes vis- especially boys, receive more criticism and pun- ible. It seems that educators have unknowingly Textbooks ishment and are more likely to be suspended absorbed the racist ideology that permeates and teaching from school than European American students. American institutions, regulations, structures, materials It has been found that teachers respond more and society, and diminishes people of color and continue to negatively to African American high-achieving women. This racist ideology is socially and purposefully constructed and maintained, to pay scant girls than European American girls (who also receive less positive attention than European the extent that educators come to accept differ- attention American boys, high-achieving or otherwise) and ential achievement as the norm and so behave to the tend to encourageand rewardsocial skills in ways that uphold this norm. Interrupting this experiences versus academic behavior on the part of African mindset and its damaging consequences re- of women American girls. When one examines the litera- quires conscious action and a deliberate change in the way classroom business is conducted. and people ture on teachers' interactions with their students, an implicit hierarchy emerges. It appears that Eu- of color. ropean American boys benefit from the most fa- Coming to Know Ways of Knowing vorable teacher interactions, followed by Euro- Numerous investigations have sought to dis- pean American girls, then girls of color, cern the mediating influence of race and cul- particularly African American girls, with African ture on children's ways of knowing and sense- American and Latino boys receiving the least making styles.6 For example, Huber and positive teacher attention. Pewewardy (1990) conducted an extensive re- Differences in curricula also affect oppor- view of research examining cultural cognitive tunities to learn. Textbooks and teaching mate- styles that concluded with the notion that dif- rials continue to pay scant attention to the ex- ferent racial and ethnic groups display numer- periences of women and people of color. ous cognitive, learning-style, interactional, and Textbook exemplars used to illustrate concepts communicative preferences.' Researchers and and topics continue to portray males and whites scholars have theorized that the differential more frequently and more advantageously than school experiences and academic achievement females and individuals of color. Additionally, of children of color may be attributed to a mis- evidence indicates that girls are more likely to match between the culture of the school and be encouraged to pursue "soft" sciences such the home cultures of pupils. This concept has as biology over high-status subjects such as been variously described as "bicultural ambiva- physics and engineering, and that boys are lence," "cultural discontinuity or incongru- more likely than girls and children of color to ence,"" and an absence of "cultural synchroni- be steered toward math- and science-related zation." These theories raise the possibility that courses. Research has also found that children culturally and linguistically diverse children may perceived as "disadvantaged" are more likely learn in culture-specific ways and require instruc- to be fed a steady diet of low-level, skill-based tion that capitalizes on their learning styles and worksuch as basic computation in mathemat- strengths, rather than emphasizing their "defi- ics and decoding in readingand relegated to cits." These theories also suggest that cognitively undemanding academic tasks. In the manner in which children of color receive, fact, children of color are disproportionately manipulate, transform, and express knowl- assigned to the lowest academic tracks, special edge, as well as their task and modality prefer- education, and the lowest-ability groups, where ences and the ways in which they interact and they are often exposed to curricula that are sim- communicate with others, may not be well ex- plified, reduced, and watered down. plained by mainstream learning theory tradi- Teachers' differential treatment of children tionally grounded in white children's ways of of color and girls has been linked to the low knowing." and limited expectations teachers have for The growing body of evidence that sup- them. Teachers perceive girls to be less academi- ports the idea of culturally grounded learning, Continued p. 4, "Ways of Knowing" 3
March 2000 WEEA Equity Resource Center at EDC, 55 Chapel Street, Newton, MA 02458-1060800-225-3088 March 2000 3 WEEA Resourceson Assessment
Expectations and beliefs in children's potential to learn playa major role in assessment and outcomes. These resources, selected from our extensive collection, can help improve classroomsystems, interactions, and outcomes for all students. They offer opportunities to infuse the experiences and perspectives of differentgroups of students and their families into the existing curriculum and to infuse equity concepts into all levels of school operation.
A-Gay-Yah An exciting, multicultural social studies or history curricu- lum for grades 6 to 12, A-Gay-Yah emphasizes critical think- Practical Tools and Support for ing and cooperative learning, increases gender equity and Gender-Fair Learning cultural awareness, and uses the context of American In- To order dian history and culture to examine gender roles. By The WEEA Equity Resource Center at EDC can Wathene Young (178 pp.) 1992 #2735 $30.00 help you find the tools you need to create gender- WEEA fair multicultural learning environments. Add-Ventures for Girls materials Building Math Confidence Call the Center's hotline at 800-225-3088 or TTY 800-354-6798 for resources and referrals. call our Developed with and field-tested by classroom teachers, this collection of fun, hands-on activities address teacher-stu- The Center's website is full of exciting distribution dent interaction patters, girls' learning styles, and the im- information and tools, from fun facts about the center at portance of parent involvement to help teachers create an history of equality to a list of practical curricula 800-793-5076. environment that engages elementary and middle school designed to help make any subject gender-fair. The girls in math. By Dr. Margaret Franklin Center's website was designed to be accessible to Elementary (292 pp.)0 #2709 $39.00 users with disabilities. Middle School (347 pp.) #2710 $42.00 www.edc.org/WomensEquity Checklists for Counteracting Race and Sex EDEQUITY (the Educational Equity Discus- Bias in Educational Materials sion List) is designed to encourage discussion For more than 15 years, this easy-to-use handbook has about international theory and practice. To helped educators and families evaluate materials for gen- subscribe, send e-mail to
WEEA Digest WEEA Equity Resource Center at EDC, 55 Chapel Street, Newton, MA02458-1060800-225-3088 Education Assessment 4
Ways of Knowing ...continued also offers insight into girls' ways of knowing.il breadth of response, cycles of revision and re- This literature suggests that girls respond more finement, students' engagement in self-assess- positively to classroom environments that en- ment, and connections between what is being courage cooperation and collaboration over the assessed and real-world issues and questions." competition, individualism, and objectivity that Standardized testing has a long history of dis- Because more typically define classroom culture. Re- advantaging girls and youngsters who are poor authentic search into girls' sense of fairness and morality or are members of visible racial / ethnic groups." assessment is posits that girls are more likely to be sensitive Thus, there is a great deal of optimistic antici- continuous, it to the needs of others, while other studies have pation, even in the face of scant evidence, that assessments that represent alternatives to tra- is integral to indicated that girls are more likely to respond to adults and to seek out interactions with them. ditional multiple-choice testing can bring about as opposed to The idea of "field sensitivity" has also been as- more equitable educational outcomes for girls divorced from sociated with females and with children of and children of color" because they enable instruction; color,13 and translates into a learning style that teachers to tailor instruction to learners and teaching and is responsive to modeling, group work, and per- truly meet their needs. sonal connections with content. assessment Authentic Assessment as a Journey become Ways of knowing notwithstanding, teach- ers would be unwise to rush to categorize chil- toward Transfomiative Teaching seamless, dren according to learning-style preference or Two basic assumptions underlie this discussion simultaneous to use these theories as rigid indicators of how about authentic assessment. First, assessment processes. culturally diverse children learn, because much is authentic when it is continuous, cyclical, and of what we know remains inconclusive and embedded in the classroom curriculum. Sec- untested.14 Rather, the lesson teachers can and ond, because authentic assessment is continuous, should take from this body of literature is the it is integral to as opposed to divorced from in- idea that children do learn and perceive the struction; teaching and assessment become seam- world in dissimilar ways. Therefore, meeting less, simultaneous processes. Rather than an the instructional and personal needs of diverse event such as testing, which happens at the con- learners demands that teachers create more in- clusion of instruction apart from the curriculum, clusive classroom cultures that embrace mul- authentic assessment is an ongoing process that tiple ways of knowing. When children are not supports and informs teaching and learning. forced to "enter school having to unlearn or, at Authentic assessments engage students in prob- least, to modify their own culturally sanctioned lem-solving and problem posing; are grounded interactional and behavioral styles and adopt in meaningful, "real-life" tasks; provide multiple those styles rewarded in the school context if forms of evidence about student learning; offer they wish to achieve academic success,"15 they students numerous opportunities for self-reflec- are allowed to apply all of themselves to the tion and revision; present varied paths to learn- educational enterprise, to bring all that they ing by encouraging the utilization of many mo- know to learning. Authentic assessment, when dalities and strategies; support children's best viewed as a way of coming to know what a per- work by rendering criteria and standards explicit; son thinks, feels, knows, and is able to do, can allow children to make connections between be used as a mechanism for revealing children home and school and to integrate different sub- and uncovering their capacities. ject areas or concepts; emphasize growth and de- Authentic assessments'6 are often described velopment over time; and value the learning pro- as more meaningful and comprehensive mea- cess as well as the product." Authentic assess- sures of what learners know and are able to do. ments require that teachers change the way they Unlike standardized tests that are efficient, think about knowledge, instruction, and aca- norm-referenced instruments that separate demic success, because achievement is no longer learning from testing and require learners to defined as getting the single right answer, doing produce distinct and correct answers on cue," things in a certain way, delivering isolated facts, authentic assessments are characterized by con- or demonstrating particular competencies on tinuous observations of learning, depth and demand. 5 Continued p. 6, "Ways of Knowing"
March 2000 WEEA Equity Resource Center at EDC, 55 Chapel Street, Newton, MA 02458-1060 800-225-3088 March 2000 Additional Resources
Achieving Gender Equity New Standards Reference Examinations Strategies for the Classroom Representing a new and exciting way to measure student This book offers strategies teachers can use to modify their achievement, performance standards indicate what stu- own classroom teaching, as well as tips for parents. Among dents should be able to do at different points in their edu- the topics addressed, the concept of self-assessment is in- cational careers. After establishing performance standards, troduced as a key challenge for women and girls; practical educators design an assessment to determine how well stu- approaches are included as well. Dianne D. Horgan (1995). dents have learned the tasks, concepts, and skills described Allyn and Bacon, 160 Gould Street, Needham Heights, MA by those standards. The New Standards Reference Examina- 02194.800-666-9433* ISBN: 020515459X tions system includes reference examinations in mathemat- Web: vig.abacon.com ics and English language arts. Includes a mix of traditional tests and performance tasks that ask students to use their ASSESS (Assessing Sex Equity in Schools knowledge to solve complex problems. Spanish version is and Society) available for the mathematics component. Harcourt Brace A useful handbook of checklists, surveys, and questionnaires Educational Measurement *800-211-8378 designed to help evaluate equity within schools. Michigan Web: www.hbem.com Center for Career and Technical Education, 230 Erickson Hall, Michigan State University, East Lansing, MI Performance-Based Student Assessment 48824.800-292-1606. Challenges and Possibilities This is Part 1 of the 95th Yearbook of the National Society for Assessment Alternatives for Diverse the Study of Education. This collection of essays discusses Classrooms the need to accurately assess students' abilities while al- This volume takes a comprehensive look at assessment in lowing for differences in what students know and how they the classroom as it affects students of color, women and learn. Addresses the intellectual, technical, and political is- girls, students with disabilities, and students of varied so- sues of reforming education assessment to accommodate cioeconomic classes. Beginning with a brief history of as- today's diverse classrooms. The essays include insight sessment from the inception of IQ testing, this book out- gained in several pilot-tested assessment systems, as well lines the bias inherent in the assessment process and as designs for new systems to assess aptitude and practical approaches toward making evaluation more ac- achievement. Edited by Joan Boykoff Baron and Dennie cessible for all students. Edited by Beverly P. Farr and Elise Palmer Wolf (1996). The University of Chicago Press, 5801 Trumbull (1997). Norwood, MA: Christopher-Gordon South Ellis, Chicago, IL 60637 773-702-7700 *ISBN: Publishers ISBN: 092684251X. 0226038033 Web: www.press.uchicago.edu Assessment for Equity and Inclusion Sex Equity in Educational Opportunity, Embracing All Our Children Achievement, and Testing A collection of essays on the importance of equity in as- Proceedings of the 1991 ETS Invitational Conference sessment, including case studies, examples, and strategies Excerpts of presentations given at the 1991 ETS Proceed- for implementing authentic assessment in the classroom. ings on measurement and evaluation. Features commen- Edited by A. Lin Goodwin (1997). Routledge, 29 West 33'h taries by national equity leaders on a wide range of topics Street, New York, NY 10001.800- 634 -7064 Web: in standardized assessment, from gender gaps in verbal and www.routledge-ny.com mathematics ability to the school and career experiences. Educational Testing Service (1992), Rosedale Road, Princeton, Assessment Standards for School NJ 085410609-921-9000 e-mail: [email protected] ISBN: Mathematics 0886851289 Web: www.ets.org Guide for examining current assessment practices and planning new assessment systems. Shows how to assess TIMSS student performance by using new approaches to deter- Third International Mathematics and Science Study mine students' progress and achievement of goals. Presents The largest study of comparative educational achievement six assessment standards that address mathematics, learn- ever undertaken, the TIMSS study compares mathematics ing, equity, openness, inferences, and coherence. National and science achievement of students in 41 countries at five Council of Teachers of Mathematics, 1906 Association grade levelsthe third, fourth, seventh, eighth grades, and Drive, Reston, Virginia 20191-1593800-235-7566ISBN: final year of secondary school. Includes charts and com- 0873534190 Web: www.nctm.org parisons by gender. Housed at Boston College, all reports are available free on -line. TIMSS International Study Cen- Enriching Content Classes for Secondary ter (1993-99), Champion Hall 323, Boston College, Chest- ESOL Students nut Hill, MA 02167 Web: www.csteep.bc.eduitimss A practical handbook for designing, adapting, and imple- menting appropriate curricula and authentic assessment for English-language learners, this book includes check- lists, student activities, and classroom ideas that can be ap- plied directly and easily.Judith H. Jameson (1998). Cen- ter for Applied Linguistics, Sunbelt Office and Delta Systems Co., Inc., McHenry, ILISBN: 1887744142 Web: delta.kksbb.com 6
WEEA Digest WEEA Equity Resource Center at EDC, 55 Chapel Street, Newton, MA 02458-1060800-225-3088 Education Assessment 6
Ways of Knowing ..continued Assessment that is authentic looks carefully tical need to assign a grade for each student in at children in order to come to know them, their math and science."23 As his conception of as- needs, and their dreams. Observing and assess- sessment began to change from "a labeling and ing students closely give teachers the precious sorting tool" to "a starting point for working cues and clues they need to create learning en- with students," he began to teach differently vironments that invite all learners in and allow to students he now perceived differently. He them to build their senses of themselves as pow- started to group students heterogeneously, to When erful, capable, and cared-for human beings. encourage students to work together, to see teachers When teachers strive to learn as much about knowledge as complex rather than discreet and learn to see students as possible and provide students with sequential. By defining "assessment as empow- erment: sketches of progress to build upon,"24 children multiple entry points to learning, they define their role as providing the supports and struc- he and his students challenged the prevailing differently tures necessary to help children see themselves norm in his schoolthat students from poor as able and as learners who are in control of the learning and working class-neighborhoods seldom willing to process. Much has been written about differ- made it into the elite subjects, specifically alge- learntheir ent kinds of authentic assessmentsportfolios, bra, that serve as gatekeepers for college entry. teaching is performances, demonstrations, exhibitions. When Julie Savitch and Leslie Serling de- These are all worthwhile activities that enable cided to team-teach their two respective transformed. students to reveal what they know in numer- classesone designated "gifted," the other ous ways. However, in the absence of deep "regular"little did they realize that their con- knowledge of students, these activities will be ceptions of giftedness would be dramatically hollow. Thus, what is most critical for teachers altered. Informed by Gardner's theory of mul- to understand when working with children in tiple intelligence,z5 they began to understand general, and diverse learners in particular, is "that education is not merely to sort out a few that "assessment is an attitude before it is a children and make them the leaders, but to method."22 Authentic assessment begins with develop the latent talents of the entire popula- teachers making it their business to purpose- tion in diverse ways."26 As their "lens for as- fully watch, listen to, talk with, and think about sessment assumed a broader view," they were the children in their classrooms. By observing, guided "to see and think about students' recording, informally monitoring, conferencing growth in different ways."27 Employing coop- with, and interviewing their students, teachers erative groups, thematic curricula, long-term initiate an ongoing process that uncovers who projects, a variety of instructional strategies, learners are and what they know, and leads to and authentic assessments such as portfolios, opportunities for teachers and children to build Savitch and Sterling invited every student into shared meaning and beliefs. But it is more than the learning process and "created a new defi- simply gathering data about children; it is al- nition of giftednessone that includes every- lowing children to get inside you so that you body."28 As a consequence of coming to know can never look at them in ways other than the their students' gifts, all their students experi- most caring and positive. enced success, including the many children When children's capacities are uncovered who were from immigrant families and spoke and they are revealed to be multifaceted learn- a language other than English, who had previ- ers, teachers' conceptions of them are naturally ously been categorized as "nongifted." challenged. When teachers learn to see children Paula "grew up with great misconceptions differentlyas able and willing to learntheir concerning what mathematics was about and teaching is transformed. Stories of teaching what it meant to be good at math." Despite transformations reveal the power of teachers' her success in mathematics, she "never felt [she] expectations and assumptions and how firmly had the right to call [herself] a math star," a la- entrenched is the sorting and classifying func- bel she felt was reserved for those who scored tion of schools. Steve Ellwood began his teach- "not high, but highest, on tests."3° As a teacher ing career as a "technician" who saw tracking of mathematics who understood "that the num- as "normative and sensible" and defined his ber of Americans who enjoy math and feel they "role in assessment [as] grounded in the prac- Continued p. 7, "Ways of Knowing" 7
March 2000 WEEA Equity Resource Center at EDC, 55 Chapel Street, Newton, MA 02458-1060800-225-3088 March 2000 7
Ways of Knowing... continued do it well is alarmingly small, and that the pre- relying unduly on single modes of expression ponderance of those are not women and mi- to the exclusion of other means. Through their norities,"" she sought to build children's un- experiences, we see that authentic assessment derstanding and enjoyment of mathematics. is dependent upon a deep belief in children's Rather than thinking of math as close-ended inherent capacity to learn and achieve academic and objective, she emphasized imagination, success, and a strong obligation to ensure edu- Teachers who personal relevance, and patterns; came to see cational equity for and access to all children engage in math as "more art than arithmetic"; and worked girls and boys, poor children, children of color. authentic "to subordinate teaching to learning."" For If schools and educators subscribe to defi- assessment Paula, "assessment was almost indistinguish- able from practice" and "at report card time cit views of children and their families and at- believe that tribute learners' shortcomings to gender, [she] sent home narratives describing how each race, different ways child worked, what her style was, what or class, they fail to be accountable for the aca- demic progress and development of all chil- of knowing strengths she relied on."" In her story, she de- dren. Much has been written about teacherex- are nurtured fines assessment in mathematics as enabling and supported children to think and do through building and pectations and the detrimental effect that low or inappropriate expectations can have on stu- concrete demonstrations; imagining and ex- by diverse dent achievement." Indeed, a growing body of trapolating from actual to imagined situations; methodologies, literature describes the relationship between writing creatively in order to encourage chil- a wide range teacher beliefs, assumptions, attitudes, andpre- dren to both reflect on their learning and gen- of activities, erate knowledge; and sharingtalking together conceptions and teacher behavior." This litera- ture tells us that teachers' belief in children's differentiated about mathematics. She advises other teachers, potential to learn and worthiness has a bearing instruction, particularly those who aim to teach math on the quality of instruction children receive. to be ready to plan and assess mathematical deep caring Authentic assessment and practice activity not through the demands of a stan- are possible only when teachers believe that children who for the dardized test but by looking at what children are culturally and linguistically diverse can and uniqueness of actually do when they think about math and must learn, and are fully capable and will ben- each child, what they say about what they do. By honor- efit from instruction that is meaningful and rich ing in their teaching and their assessing, the and the with powerful ideas. Too often, childrenare creation of an multiplicity of ways that children talk and blamed for their own failures think and explain their math, teachers will be or locked into teachers' assumptions about what they inviting rehumanizing the discipline.34 can and cannot accomplish. Teachers can and will find classroom A Final Word on Authentic many convenient reasons for children's lack of family. success unless they look closely and critically Assessment at themselves, scrutinize their own practices, These portraits of practice enable us to see that and abide by the assumption that if children teachers who engage in authentic assessment are not learning, then the teachers are the ones believe that different ways of knowing are nur- who must do something differently. This isnot tured and supported by diverse methodologies, easy and requires "a very special kind of lis- a wide range of activities, differentiated instruc- tening, listening that requires not onlyopen tion, deep caring for the uniqueness of each eyes and ears, but open hearts and minds. We child, and the creation of an inviting classroom do not really see through our eyesor hear family." Each of the teachers previously intro- through our ears, but through our beliefs."" duced assumes that children possess much Notes knowledge; each sees the purpose of instruc- 1. In this article, I use the term of color to denote those in- tion and assessment as first supporting learn- dividuals who are African American, Asian American, ers to reveal what they know and then guiding Latino, and Native American, or who are not of Euro- them to compare, relate, or apply what they pean American descent, even while I acknowledge that know to new information and experiences; and all human beings are, in actuality, people of color. each is determined to educate every child. Each 2. See for example, American Association of University Women (AAUW), How Schools Shortchange Girls. (Wash- works to enable his or her students to demon- ington, DC: American Association of University Women strate knowing in a variety of ways and resists and National Education Association, 1992). College Continued p. 8, "Ways of Knowing" 8 WEEA Digest WEEA Equity Resource Center at EDC, 55 Chapel Street, Newton,MA 02458-1060800-225-3088 8 Education Assessment
Ways of Knowing... continued Board, Equality and Excellence: The Educational Status of A. Castaneda, Cultural Democracy, Bicognitive Develop- Black Americans (New York: College Board, 1985). L. Dar- ment, and Education (New York: Academic Press, 1974). ling-Hammond, "Inequality and Access to Knowledge." 14. R. T. Carter and A. L. Goodwin, "Racial Identity and In J. A. Banks (Ed.), Handbook of Research on Multicultural Education." In L. Darling-Hammond (Ed.), Review of Re- Education (New York: Macmillan, 1995): 465-483. D. Pol- search in Education, Vol. 20 (Washington, DC: American lard, "Gender and Achievement." In S. K. Biklen and Educational Research Association, 1994): 291-336. G. D. Pollard (Eds.), Gender and Education: Ninety-second Ladson-Billings, "Culturally Relevant Teaching: The Yearbook of the National Society for the Study of Education Key to Making Multicultural Education Work." In C. (Chicago, IL: University of Chicago Press, 1993): 90-106. A. Grant (Ed.), Research and Multicultural Education (Lon- 3.L. Delpit, Other People's Children (New York: New Press, don: Falmer, 1992): 106-121. 1995). A. L. Goodwin, (Ed.), Assessment for Equity and In- 15. S. Fordham, "Racelessness as a Factor in Black Students' clusion: Embracing All Our Children (New York: Routledge, School Success: Pragmatic Strategy or Pyrrhic Victory?" 1997a). J. J. Irvine, Black Students and School Failure (New Harvard Educational Review, 58 (1988): 55. York: Praeger, 1991). G. Ladson-Billings, The Dreamkeepers 16. This article uses the term authentic assessment to gener- (San Francisco, CA: Jossey-Bass, 1994). S. Nieto, Affirm- ally depict holistic assessments that are embedded in ing Diversity (New York: Longman, 1992). V. 0. Pang classroom contexts and enable children to demonstrate and L. L. Cheng, Struggling to Be Heard: The Unmet Needs learning by integrating and applying knowledge and of Asian Pacific American Children (Albany, NY: SUNY skills to real-world tasks. However, it is important to ac- Press, 1998). P. G. Ramsey, Teaching and Learning in a Di- knowledge that a variety of terms exist in the literature verse World (New York: Teachers College Press, 1987). that are either used interchangeably with or are presented 4., A. L. Goodwin, "The Social /Political Construction of as examples of authentic assessment. These include per- Low Teacher Expectations for Children of Color: How formance assessment, alternative assessment, portfolio assess- Can We Learn to See with Different Eyes?" In S. H. King ment, naturalistic assessment, situated assessment, dynamic (Ed.), The Thought and Practice of Anti-Racist Teaching assessment, curriculum-embedded assessment, and assessment (New York: Routledge, forthcoming). by exhibition (Garcia & Pearson, 1994). 5. R. T. Carter and A. L. Goodwin, "Racial Identity and 17. G. P. Wiggins, Assessing Student Performance. (San Fran- Education." In L. Darling-Hammond (Ed.), Review of Re- cisco: Jossey Bass, 1993). search in Education, Vol. 20 (Washington, DC: American 18. L. Einbender and D. Wood, An Authentic Journey: Teach- Educational Research Association, 1994): 319. ers' Emergent Understandings about Authentic Assessment 6. R. Dunn and S. A. Griggs, "Research on the Learning and Practice (New York: National Center for Restructur- Style Characteristics of Selected Racial and Ethnic ing Education, Schools, and Teaching, 1995). Groups." Journal of Reading, Writing, and Learning Dis- 19. American Association of University Women (AAUW), abilities International, 6, (1990): 261-280. G. Gay, "Cul- How Schools Shortchange Girls. (Washington, DC: Ameri- turally Diverse Students and Social Studies." In J. P. can Association of University Women and National Shaver (Ed.), Handbook of Research on Social Studies Edu- Education Association, 1992). G. E. Garcia and P. D. cation (New York: Macmillan, 1991): 144-156. J. J. Irvine, Pearson, "Assessment and Diversity." In L. Darling- Black Students and School Failure (New York: Praeger, Hammond (Ed.), Review of Research in Education, Vol. 20 1991). B. J. Shade, "Afro-American Cognitive Style: A (Washington, DC: American Educational Research As- Variable in School Success?" Review of Educational Re- sociation 1994): 337-392. S. J. Gould, The Mismeasure of search, 52, (1982): 219-244. Man (New York: Norton, 1981). J. Oakes, Keeping Track: 7.T. Huber and C. Pewewardy, "Maximizing Learning for How Schools Structure Inequality (New Haven, CT: Yale All Students: A Review of Literature on Learning Mo- University Press, 1985). dalities, Cognitive Styles, and Approaches to Meeting 20. E. W. Gordon and C. Bonilla-Bowman, "Can Perfor- the Needs of Diverse Learners" (1990). (ERIC Document mance-Based Assessments Contribute to the Achieve- Reproduction Service, No. ED 324 289.) ment of Educational Equity?" In J. B. Baron and D. P. 8.J. Cummins, "Empowering Minority Students: A Frame- Wolf (Eds.), Performance-Based Student Aslessment: Chal- work for Intervention. Harvard Educational Review, 56 lenges and Possibilities (Ninety-fifth Yearbook of the National (1986): 18-36. Society for the Study of Education, Part I, Chicago, IL: 9. S. Nieto, Affirming Diversity (New York: Longman, 1992). University of Chicago Press, 1996): 32-51. 10. J. J. Irvine, Black Students and School Failure (New York: 21. L. Darling-Hammond and B. Falk, "Supporting Teach- Praeger, 1991). ing and Learning for All Students: Policies for Authen- 11. R. T. Carter and A. L. Goodwin, "Racial Identity and tic Assessment Systems." In A. L. Goodwin, (Ed.), As- Education." In L. Darling-Hammond (Ed.), Review of Re- sessment for Equity and Inclusion: Embracing All Our Chil- search in Education, Vol. 20 (Washington, DC: American dren (New York: Routledge,1997): 51-76. C. Genishi, Educational Research Association, 1994): 319. "Assessing against the Grain: A Conceptual Framework 12. American Association of University Women (AAUW), for Alternative Assessments." In A. L. Goodwin, (Ed.). How Schools Shortchange Girls (Washington, DC: Ameri- Assessment for Equity and Inclusion: Embracing All Our can Association of University Women and National Edu- Children (New York: Routledge, 1997): 35-50. Y. Smith cation Association, 1992). C. Gilligan, In a Different Voice: and A. L. Goodwin, "The Democratic, Child-centered Psychological Theory and Women's Development (Cam- Classroom: Provisioning for a Vision." In A. L. Goodwin bridge, MA: Harvard University Press, 1982). (Ed.), Assessment for Equity and Inclusion: Embracing All 13. H. Grossman and S. H. Grossman, Gender Issues in Edu- Our Children (New York: Routledge, 1997): 101-120. cation (Boston: Allyn Sr Bacon, 1994). M. Ramirez III and Continued p. 9, "Ways of Knowing" 9
March 2000 WEEA Equity Resource Center at EDC, 55 Chapel Street, Newton, MA 02458-1060800-225-3088 March 2000 9 Resource Organizations for Assessment
The College Board The National Center for Fair & Open Testing (FairTest) 45 Columbus Avenue 342 Broadway New York, NY 10023-6992 Cambridge, MA 02139 Phone: 212-713-8000 Phone: 617-864-4810 Web: www.collegeboard.org E-mail: [email protected] Council of Chief State School Officers Web: www.fairtest.org One Massachusetts Avenue, NW, Suite 700 National Center for Improving Student Learning and Washington, DC 20001 Achievement in Mathematics and Science Phone: 202-408-5505 Wisconsin Center for Education Research Web: www.ccsso.org School of Education Educational Testing Service University of Wisconsin-Madison Rosedale Road 1025 W. Johnson Street Princeton, NJ 08541 USA Madison, WI 53706 Phone: 608-263-3605 Phone: 609-921-9000 E-mail: [email protected] Web: www.wcer.wisc.edu/ncisla Web: www.ets.org National Center for Research and Evaluation, Standards, ERIC Clearinghouse on Assessment and Evaluation and Student Testing 1131 Shriver Laboratory (Bldg 075) UCLA Center for the Study of Evaluation University of Maryland CSE /CRESST College Park, MD 20742 301 GSE&IS Mailbox 951522 Phone: 301-405-7449 or 800-464-3742 Los Angeles, CA 90095 E-mail: [email protected] Phone: 310-206-1532 Web: www.ericae.net Web: www.cse.ucla.edu Learning Through Evaluation, Adaptation, and Northwest Regional Educational Laboratory Dissemination (LEAD) Center Assessment and Evaluation Program 1402 University Ave. 101 SW Main, Suite 500 Madison, WI 53706 Portland, OR 97204 Phone: 608-265-5943 Phone: 503-275-9500 E-mail: [email protected] Web: www.nwrel.org/eval Web: www.cae.wisc.edu/-lead
Ways of Knowing... continued
22. E. Chittenden, "Authentic Assessment, Evaluation, and 33. Ibid, p. 134 Documentation." In V. Perrone (Ed.), Expanding Student 34. Ibid, p. 138 Assessment (Alexandria, VA: Association for Supervision 35. W. W. Purkey and J. M. Novak, Inviting School Success and Curriculum Development, 1991): 29. (2nd ed.). (Belmont, CA: Wadsworth, 1984). 23. S. C. Ellwood IV, "The Power of Possibilities." In A. L. 36. J. L. Brophy and T. L. Good, "Teacher Behavior and Stu- Goodwin (Ed.), Assessment for Equity and Inclusion: Embrac- dent Achievement." In M. C. Wittrock (Ed.), Handbook ing All Our Children (New York: Routiedge, 1997): 77-100. of Research on Teaching (3rd ed.). (New York: Macmillan, 24. Ibid, p. 85 1986): 328-375. T. L. Good and J. Brophy, Looking in Class- 25. H. Gardner, Frames of Mind (New York: Basic Books, 1983). rooms (3rd. ed.). (New York: Harper Sr Row, 1984). J. J. 26. Winn, 1990 cited in J. H. Savitch & L. A. Serling, "I Irvine, Black Students and School Failure (New York: Wouldn't Know I Was Smart If I Didn't Come to this Praeger, 1991). Class." In A. L. Goodwin (Ed.), Assessment for Equity and 37. C. M. Clark and P. L. Peterson, "Teachers' Thought Pro- Inclusion: Embracing All Our Children (New York: cesses." In M. C. Wittrock (Ed.), Handbook of Research on Routledge, 1997): 141-162. Teaching (3rd ed.). (New York: Macmillan, 1986): 255- 27. Ibid, p. 155 296. A. L. Goodwin, "Historical and Contemporary Per- 28. Ibid, p. 159 spectives on Multicultural Teacher Education: Past Les- 29. P. Hajar, "Awakening to the Mathematician Within: One sons, New Directions." In J. King, E. R. Hollins and W. Teacher's Story." In A. L. Goodwin (Ed.), Assessment for Hayman (Eds.), Meeting the Challenge of Diversity in Equity and Inclusion: Embracing All Our Children (New Teacher Preparation (New York: Teachers College Press, York: Routledge, 1997): 121-140. 1997b): 5-22. F. A. Rios, Teacher Thinking in Cultural Con- 30. 'bid, p. 123 texts (Albany, NY: SUNY Press, 1996). 31. Ibid, p. 123 38. L. Delpit, Other People's Children (New York: New Press, 32. Ibid, p. 124 1995): 46.
10 BEST COPY AVAILABLE
WEEA Digest WEEA Equity Resource Center at EDC, 55 Chapel Street, Newton, MA 02458-1060800-225-3088 10 Education Assessment
Academic Accountability . .continued ability for academic progress while meeting standard. They then wrote one or two sample individual student needs. The question for performance activities to give teachers ideas of states and localities then becomes, What alter- how to structure classroom assessments di- nate assessments are appropriate and how are we to rected toward the APIs. The groups used an develop them in ways that meet the enormous di- organizational chart with four columns under Every state in versity represented within these two special-needs each content standard. The left-hand column the nation populations? Every state in the nation must ad- listed the corresponding performance stan- must address dress this issue by July 2000, when the federal dards, followed by a column for the draft APIs, this issue by reporting requirements take effect. The ap- then the sample draft activities / tasks, and fi- July 2000, proaches we see may be almost as varied as the nally a blank column that provides teachers number of states in the Union. Wisconsin's an- with space to document their sources of assess- when the swer is a complex response that attempts to ment data (e.g., work samples, direct observa- federal meet the spirit of the legislation, the needs of tion, review of records, tests).4 To accompany reporting these students, and the tradition of local con- the APIs, alternative assessment guidebooks requirements trol and autonomy that is carefully guarded in detailing their appropriate use were developed. take effect this Midwest state. The guidebook for teachers of students with LEP demonstrates how to design and use APIs, Creating a Framework including numerous examples of how to imple- The key component of Wisconsin's approach ment performance-based classroom assess- is the state's framework for classroom-based, ments. It also assists educators in creating as- alternative assessment for students who can- sessment rubrics, interpreting data, measuring not meaningfully participate, even with allow- gains over time, and reporting results at the able accommodations, in the regular assess- local level. The guidebook for teachers of stu- ment program. The framework is aligned with dents with disabilities includes extensive infor- Wisconsin's model academic standards, which mation on using testing accommodations. Both describe what students should know and be guidebooks will be used as the foundational able to do by grades 4, 8, and 12. These stan- texts in professional development sessions that dards include broad content objectives followed have already begun in Wisconsin. by several more focused performance bench- The guidebooks also provide suggestions marks for progress in the four academic sub- for how schools can report student progress in ject areas of social studies, language arts, math, alternative assessment locally to parents and and science. To create the framework, the Wis- the community. Students taking alternative as- consin Department of Public Instruction called sessment are included in statewide reports, together educators from around the state to the along with students participating in standard- capital, Madison, in summer 1998 to develop ized assessments. In this way, all students with Alternate Performance Indicators (APIs). APIs are disabilities and LEP "count" in building per- observable, measurable indicators of progress formance reports and thus cannot "disappear" toward meeting particular content and perfor- from the accountability equation. While the mance standards. The APIs assist teachers who expectation for students with LEP is that alter- work with students with disabilities or LEP in native assessment is a temporary need while accessing the state's academic standards by English skills develop, students with severe providing examples of concrete progress indi- disabilities may participate in alternative as- cators and practical assessment activities. sessment as long as the Individualized Educa- The educators who developed APIs were tional Plan (IEP) Committee deems such par- first grouped separately by their specialty of ticipation appropriate.' working with either students with disabilities or LEP. Once in these groups, the teams were Advantages of the Alternative further divided into the four academic content Assessment Framework areas. Their charge was to consider the special A standards-based, alternative assessment needs of the students they taught while review- framework assists teachers in planning lessons ing each performance standard, writing be- and assessments aligned with the same high tween one and three APIs for each performance Continued p. 11, "Academic Accountability" 11
March 2000 WEEA Equity Resource Center at EDC, 55 Chapel Street, Newton, MA 02458-1060800-225-3088 March 2000 11
Academic Accountability... continued standards other students must meet. This align- Notes ment between standards-based curricula, in- 1.While many prefer to call these students English-lan- struction, and assessment is particularly impor- guage learners, I use students with Limited English Pro- tant for a group of students who have often ficiency (LEP), as this is still the term used in federal been denied access to quality academic content. and state legislation. In this sense, the APIs serve as much as a cur- 2. D. August and K. Hakuta, (Eds.) Improving Schooling for Language- Minority Children: A Research Agenda (National riculum and instructional planning guide as Research Council, Washington, DC: National Academy they do an assessment framework. APIs pro- Press, 1997). L. M. McDonnell, M. J. McLaughlin and P. mote multiple ways of assessing LEP student Morrison, (Eds.) Educating One and All: Students With performance that are authentic and take place Disabilities and Standards-Based Reform (National Re- search Council, Washington, D.C.: National Academy over time. This is congruent with the best-prac- Wisconsin tice recommendations for the assessment of Press, 1997). J. Olsen and A. Goldstein, The Inclusion Of Students With Disabilities And Limited English Proficient can now special-needs students .° Content validity is high Students In Large-Scale Assessments: A Summary Of Re- with APIs, as they are directly linked to the cent Progress (Washington, DC: U.S. Dept. of Education, say that all same academic standards other students are Office of Educational Research and Improvement, 1997). students are learning. Few if any high stakes (e.g., retention- 3. Examples of commonly used testing accommodations include taking the test with additional time, in separate included and in-grade, graduation) are attached to perfor- testing locations, with additional breaks, with dictio- that all mance on APIs, and so the issue of negative naries or other educational aids, and in large-print or students consequences from the test is less a concern.' Braille editions. count Teachers within academic support pro- 4. API Taskforce (in press), Alternate Performance Indicators for Limited English Proficient Students. Madison, WI: Wis- grams have traditionally experienced difficulty consin Department of Public Instruction. Also, sample in moving beyond separate, remedial curricula. in Assessment section of www.edc.org/WomensEquity. APIs offer teachers of students with disabilities 5. Stephen Elliott (in press), Educational Assessment and Ac- and LEP a local framework within which they countability for All Students: Facilitating the Meaningful are encouraged to align their curricula, instruc- Participation of Students With Disabilities in District and Statewide Assessment Programs. Madison, WI: Wisconsin tion, and assessment with challenging content Department of Public Instruction. M. Gottlieb (in press). and performance standards from the very be- Standards-Based Alternate Assessment for Limited English ginning. This should enable support programs Proficient Students: A Guide for Wisconsin Educators. Madi- to accelerate the rate at which these students son, WI: Wisconsin Department of Public Instruction. close the academic gap. 6. M. LaCelle-Peterson and C. Rivera, "Is It Real for All Kids? A framework for equitable assessment policies for While I would not want to give the impres- English language learners." Harvard Educational Review, sion that the alternative assessment framework 64(1), (1994): 55-75. D. August and K. Hakuta, (Eds.) will resolve all the difficulties inherent in teach- Improving Schooling for Language-Minority Children: A Re- search Agenda (National Research Council, Washington, ing such a diverse group of students, it is a sig- DC: National Academy Press, 1997). L.M. McDonnell, nificant step forward. With the development of M.J. McLaughlin and P. Morrison, (Eds.) Educating One a standards-based, alternative assessment sys- & Ali: Students With Disabilities And Standards-Based Re- tem, local schools now have a tool for curricu- form (National Research Council, Washington, D.C.: Na- lum alignment and a continuum of assessment tional Academy Press, 1997). J. Olsen and A. Goldstein, The Inclusion of Students With Disabilities and Limited En- options within which to include all students, glish Proficient Students in Large-Scale Assessments: A Sum- even those with minimal English proficiency mary of Recent Progress (Washington, DC: U.S. Dept. of or cognitive disabilities. This continuum moves Education, Office of Educational Research and Improve- from full participation in large-scaleassess- ment, 1997). ment, through participation with varying test- 7. S. Messick, "The Interplay of Evidence and Conse- quences in the Validation of Performance Assessments." ing accommodations, to, for a small percentage Educational Researcher, 23(2), (1994): 13-23. J. Olsen and of eligible students, participation in classroom- A. Goldstein, The Inclusion of Students With Disabilities based, standards-referenced alternative assess- and Limited English Proficient Students in Large-Scale As- ments. Wisconsin can now say that all students sessments: A Summary of Recent Progress (Washington, are included and that all students count. + DC: U.S. Dept. of Education, Office of Educational Re- search and Improvement, 1997).
12
WEEA Digest WEEA Equity Resource Center at EDC, 55 Chapel Street, Newton,MA 02458-1060800-225-3088 Education Assessment 12 Academic Accountability for Students with Disabilities and LEP By Tim Boals, Wisconsin Department of Public Instruction
Traditionally, many students with disabilities ing states and localities to include all students The very and LEP have been excluded from large-scale in state and local reporting of academic assess- students most academic assessments on the grounds that ment results. In this new era of increased account- in need of those assessments were inappropriate and thus ability, all students, and the schools and programs inequitable. While many researchers and prac- that serve them, must meet clearly defined, stan- accountability titioners still voice concerns about overreliance dards-referenced criteria for learning. and on these assessments, increasingly educators Under IASA and IDEA, it is no longer suffi- educational have come to believe that exclusion from large- cient to report that a child was exempted from reforms are scale assessments has more drawbacks than academic assessments. Nonetheless, the legisla- often left out benefits. Since, for better or worse, large-scale tion also recognizes that, for a small percentage assessments constitute a central piece of school of students with disabilities or LEP, large-scale accountability, the very students most in need assessments, even with testing accommodations,3 of accountability and educational reforms are will not provide an opportunity for students to often left out of the equation if they are not as- demonstrate what they know and are able to do. sessed for academic progress.' Recent federal The legislation states that for these students, al- legislation, the Improving America's Schools ternate assessments must be developed and Act (IASA) of 1994, and the Individuals with implemented to provide the required account- Disabilities Education Act (IDEA) as amended in 1997, sought to address this issue by requir- Continued p. 10, "Academic Accountability"
WEEA Equity Resource Center Nonprofit Organization Education Development Center, Inc. U.S. Postage 55 Chapel Street, D-0300 PAID Boston, MA Newton, MA 02458-1060 Permit No. 58241 617-969-7100800-225-3088 TTY: 800-354-6798 Fax: 617-332-4318 E-mail: [email protected] www.edc.org / WomensEquity
© 2000 WEEA Equity Resource Center at EDC The WEEA Digest is published by WEEA Equity Resource Center, a project at Education Development Center, Inc., under contract #ED-98-CO-0008 from the US. Department of Education. Opinions expressed herein do not necessarily reflect the position of the U.S. Department of Education and no official endorsement should be inferred.
WEEA is the Women's Educational Equity Act federal legislation to promote educational equity for girls and women.
Katherine Hanson, Senior Project Director Julia Potter, Managing Editor Sundra Flansburg, Project Coordinator Susan J. Smith, Director of Technical Assistance BEST COPY AVAILABLE Paula Fleming, Marketing Communications Director Susan Carter, Information Specialist Ambika Kapur, Research Assistant 13 Cynthia Newson, Administrative Assistant Ronnie DiComo, Production Assistant U.S. Department of Education Office of Educational Research and Improvement (OERI) National Library of Education (NLE) ERIC Educational Resources Information Center (ERIC)
NOTICE
Reproduction Basis
This document is covered by a signed "Reproduction Release (Blanket)" form (on file.within the ERIC system), encompassing all or classes of documents from its source organization and, therefore, does not require a "Specific Document" Release form.
This document is Federally - funded, or carries its own permission to reproduce, or is otherwise in the public domain and, therefore, may be reproduced by ERIC without a signed Reproduction Release form (either "Specific Document" or "Blanket").
EFF-089 (3/2000)