Academic Bias in Language Testing: a Construct Validity Critique of the IPT I Oral Grades K-6 Spanish Second Edition (IPT Spanish)
Total Page:16
File Type:pdf, Size:1020Kb
86 Academic Bias in Language Testing: A Construct Validity Critique of the IPT I Oral Grades K-6 Spanish Second Edition (IPT Spanish) Jeff MacSwan Arizona State University Kate Mahoney State University of New York Abstract Construct validity concerns for the IPT I Oral Grades K-6 Spanish Second (GLWLRQ ,376 DVDPHDVXUHRIQDWLYHRUDOODQJXDJHSUR¿FLHQF\DUHH[DPLQHG 7KHH[DPLQDWLRQLQFOXGHGGHVFULELQJDVXEVHWRILWHPVWKDWFRQWULEXWHVPRVW WRRYHUDOOVFRUHDQGQDWLYHODQJXDJHSUR¿FLHQF\GHVLJQDWLRQ&RUUHODWLRQV EHWZHHQWKLVVXEVHWRILWHPVDQGWKHRYHUDOOVFRUHDUHFDOFXODWHG(PSLULFDO evidence suggests that if a student cannot or does not answer four questions from the second section in a complete sentence, then the student will either EHODEHOHGDVDQRQVSHDNHURUDOLPLWHGVSHDNHULQWKHQDWLYHODQJXDJH7KH authors argue that the requirement that speakers respond in complete sentences UHÀHFWVDQDwYHYLHZRIODQJXDJHSUR¿FLHQF\LQFRQVLVWHQWZLWKOLQJXLVWLF research, and characterize the requirement as “academic bias,” that is, a prejudice resulting from a confusion of academic content knowledge related to language arts with actual linguistic ability, having the result of arbitrarily IDYRULQJPHPEHUVRIWKHHGXFDWHGFODVVHV7KHDXWKRUVFRQFOXGHWKDWWKH,376 should not be used for Spanish language assessment with Spanish-speaking FKLOGUHQ Introduction A number of teachers, policy makers and educational researchers have accepted, propagated, and even institutionalized the belief that some VFKRRODJHFKLOGUHQKDYHQRODQJXDJH,QWKH/RV$QJHOHV8QL¿HG6FKRRO 87 District, for instance, the Los Angeles Times reported that 6,800 children were FODVVL¿HGDV³QRQQRQV´DQGVDLGWREH³QRQYHUEDOLQERWK(QJOLVKDQGWKHLU QDWLYHODQJXDJH´ 3\OH &KLOGUHQDUHVRFODVVL¿HGDVDUHVXOWRIQDWLYH ODQJXDJHDVVHVVPHQWUHTXLUHGRUUHFRPPHQGHGIRUQRQ(QJOLVKVSHDNHUVLQ PDQ\SDUWVRIWKH86 We begin with an overview of existing state policies for the native ODQJXDJHWHVWLQJRI(QJOLVKODQJXDJHOHDUQHUVLQWKH86DQGWKHQSUHVHQWDQG critique the dominant paradigm in language minority education, the Threshold +\SRWKHVLVDQGDVVRFLDWHG%,&6&$/3GLVWLQFWLRQDFRQFHSWXDOIUDPHZRUN consistent with native oral language testing practices. As a follow-up to 0DF6ZDQDQG5ROVWDG DQLQWHUQDOYDOLGLW\DQDO\VLVRIWKH6SDQLVK YHUVLRQRIWKH,'($3UR¿FLHQF\7HVW ,376 $PRUL 'DOWRQ D ZLGHO\XVHGQDWLYHODQJXDJHDVVHVVPHQWLVSUHVHQWHGWRVKRZWKDWWKH³QRQ´ RU³OLPLWHG´SUR¿FLHQF\UDWLQJPD\IROORZIURPFRQVWUXFWLUUHOHYDQWIDFWRUV – such as knowledge of school and school culture, in the present instance. It is concluded that, in light of the validity concerns and the lack of relevant HPSLULFDOHYLGHQFHIRUWKHH[LVWHQFHRI³QRQQRQV´DVZHOODVWKHSRVVLEOH negative effects of the label itself, educational agencies should abandon the SUDFWLFHRIURXWLQHO\DVVHVVLQJFKLOGUHQ¶VRUDOQDWLYHODQJXDJHSUR¿FLHQF\ Native Language Assessment Policy in U.S. Schools $QLPSRUWDQWUHVSRQVLELOLW\RIVFKRROVLQWKH86LVWRGHWHUPLQH ZKHWKHURUQRWDFKLOGNQRZV(QJOLVKVXI¿FLHQWO\ZHOOWRVXFFHHGLQDQDOO (QJOLVKLQVWUXFWLRQDOVHWWLQJ,Q/DXY1LFKROV WKH866XSUHPH &RXUWLQWHUSUHWHG7LWOH9,RIWKH&LYLO5LJKWV$FWWRSURKLELWGLVFULPLQDWLRQ against language-minority children by means of schools failing to provide for their special language-related needs. Thus, schools must determine, for any child enrolling in school, whether the child requires special assistance to XQGHUVWDQGVFKRROFRQWHQW6WDWHVWKDWGRQRWKDYHODZVSURKLELWLQJWKHXVH RIODQJXDJHVRWKHUWKDQ(QJOLVKLQVFKRRODUHDEOHWRSURYLGHFKLOGUHQZLWK bilingual instruction, making academic content comprehensible during the WLPHQHHGHGWROHDUQ(QJOLVK $QHYDOXDWLRQRIFKLOGUHQ¶V(QJOLVKDELOLW\LVDSSURSULDWH±LQGHHG imperative – in light of these considerations. However, many states also require or recommend assessment of children’s native language ability, with the result Fall 2008 / Volume 8, Number 2 88 WKDWQXPHURXVFKLOGUHQDUHLGHQWL¿HGDV³QRQQRQV´±WKDWLVQRQVSHDNHUVRI (QJOLVKDQGQRQVSHDNHUVRIWKHLUKRPHODQJXDJH W\SLFDOO\6SDQLVKLQWKH 86FRQWH[W DFRQGLWLRQKLVWRULFDOO\NQRZQDV³VHPLOLQJXDOLVP´ &XPPLQV 6NXWQDEE.DQJDV 7DEOHOLVWVVWDWHVWKDWUHTXLUHRUUHFRPPHQG QDWLYHODQJXDJHWHVWLQJDQG7DEOHSUHVHQWVWKHPRVWIUHTXHQWO\XVHGWHVWV RIQDWLYHODQJXDJHDELOLW\ DVRIVHH0DKRQH\ 0DF6ZDQ 7DEOH States which require or recommend native language assessment for English /DQJXDJH/HDUQHUV (//V DVSDUWRIWKHLGHQWL¿FDWLRQSURFHVV 0DKRQH\ MacSwan, 2005) and the number of ELLs in each of these states (Kindler, 2002), as of 1999-2000 Total State Percentage of Total State ELL Students Enrollment State Enrollment Arizona &RQQHFWLFXW 'LVWULFWRI&ROXPELD Hawaii Illinois Mississippi Ohio Oklahoma Oregon 6RXWK'DNRWD Texas 9LUJLQ,VODQGV 9LUJLQLD Total 1,043,614 12,482,502 Rethinking the Dominant Paradigm in Language Minority Education The practice of assessing the native language of bilingual children, LIQRWGLUHFWO\GHULYHGIURPWKH7KUHVKROG+\SRWKHVLVDQG%,&6&$/3 IUDPHZRUN &XPPLQV LVFRQFHSWXDOO\VXSSRUWHGE\ it. The Threshold Hypothesis posits that children may enter into a state of ³GRXEOHVHPLOLQJXDOLVP´EURXJKWDERXWE\DSURFHVVRIQDWLYHODQJXDJH Journal of Educational Research & Policy Studies 7DEOH Tests Used by States for Native Language Assessment (Mahoney & MacSwan, 2005) Number of Instrument 6WDWHV /DQJXDJH$VVHVVPHQW6FDOH6SDQLVK /$6 ,GHD3UR¿FLHQF\7HVW6SDQLVK ,37 :RRGFRFN0XQR]6SDQLVK %LOLQJXDO,QYHQWRU\RI1DWXUDO/DQJXDJH %,1/ %LOLQJXDO6\QWD[0HDVXUH6SDQLVK %60 3HDERG\3LFWXUH9RFDEXODU\7HVW6SDQLVK 3397 subtraction, as a result of contact with a second language in the early years. 6HPLOLQJXDOLVPDOVRNQRZQDV³OLPLWHGELOLQJXDOLVP´ &XPPLQV KDV EHHQGH¿QHGLQYDULRXVZD\VEXWLWLVPRVWFRPPRQO\FRQVLGHUHGWRLQGLFDWH ³ORZOHYHOVLQERWKODQJXDJHV´RU³OHVVWKDQQDWLYHOLNHFRPPDQGRIWKH YRFDEXODU\DQGV\QWDFWLFVWUXFWXUHV´RIHLWKHUODQJXDJH &XPPLQVS &XPPLQV GH¿QHGWKH7KUHVKROG+\SRWKHVLVDVIROORZV Negative cognitive and academic effects are hypothesized to result from low levels of competence in both languages or what 6FDQGLQDYLDQUHVHDUFKHUV HJ+DQVHJDUG>@6NXWQDEE .DQJDV 7RXNRPDD KDYHWHUPHG³VHPLOLQJXDOLVP´RU ³GRXEOHVHPLOLQJXDOLVP´«(VVHQWLDOO\WKHORZHUWKUHVKROG level of bilingual competence proposes that bilingual children’s FRPSHWHQFHLQDODQJXDJHPD\EHVXI¿FLHQWO\ZHDNDVWRLPSDLU the quality of their interaction with the educational environment WKURXJKWKDWODQJXDJH S Although the Threshold Hypothesis is widely accepted, no relevant HYLGHQFHKDVEHHQSUHVHQWHGWRVXSSRUWWKH³VHPLOLQJXDOLVP´LGHDHPEHGGHG ZLWKLQLW3DXOVWRQ IRULQVWDQFHUHYLHZHGQXPHURXV6FDQGLQDYLDQ studies which sought linguistic evidence for the existence of semilingualism LQ6ZHGHQDQGFRQFOXGHGWKDW³WKHUHLVno empirical evidence to support the H[LVWHQFHRIVXFKDODQJXDJHGHYHORSPHQWKLDWXVDV³>VHPLOLQJXDOLVP@´ S HPSKDVLVLQRULJLQDO 0DF6ZDQ UHYLHZHGUHSXWHGHYLGHQFHIURP Fall 2008 / Volume 8, Number 2 studies of language variation, linguistic structure, school performance, and language loss, and concluded that all of it was either spurious or irrelevant. 0DF6ZDQ HPSKDVL]HGWKDWVHPLOLQJXDOLVPDVSUHVHQWHGE\LWV proponents, is indistinguishable from classical linguistic prescriptivism. The semilingualism idea contrasts markedly with empirically grounded ZRUNRQFKLOGODQJXDJH$V&KRPVN\ REVHUYHGORQJDJR A consideration of the character of the grammar that is acquired, the degenerate quality and narrowly limited extent of the available data, the striking uniformity of the resulting grammars, and their independence of intelligence, motivation and emotional state, over wide ranges of variation, leave little hope that much of the structure of language can be learned by an organism initially uninformed DVWRLWVJHQHUDOFKDUDFWHU S &KRPVN\¶VVWDWHPHQWSDLQWVDSLFWXUHRIFKLOGUHQDVLQZDUGO\GULYHQODQJXDJH learners who acquire their language perfectly and without instruction. During WKHPRVWDFWLYHDFTXLVLWLRQSHULRG DJHV FKLOGUHQOHDUQDSSUR[LPDWHO\WHQ to twelve new words a day, often on one exposure and in highly ambiguous FLUFXPVWDQFHV *OHLWPDQ /DQGDX &KLOGUHQNQRZWKLQJVDERXW elementary aspects of sentence structure for which they have no evidence at DOO 3LQNHU DQGLQFDVHVRIFUHROL]DWLRQFKLOGUHQDFTXLUHV\QWDFWLFDOO\ and morphologically complex linguistic systems in accordance with universal principles of grammar in the presence of highly degenerate, rudimentary adult ODQJXDJHLQSXW SLGJLQV %LFNHUWRQ In an extensive review of research on child language in the preschool \HDUV7DJHU)OXVEHUJ UHSRUWHGWKDW³E\WKHWLPHFKLOGUHQEHJLQVFKRRO they have acquired most of the morphological and syntactic rules of their ODQJXDJH´DQGSRVVHVVDJUDPPDUHVVHQWLDOO\LQGLVWLQJXLVKDEOHIURPDGXOWV S 7KLVYLHZUHSUHVHQWVDFRQVHQVXVDPRQJUHVHDUFKHUVLQFKLOGODQJXDJH acquisition, where the matter has been investigated empirically for about three GHFDGHV*LYHQWKHVHIDFWVLWLVUHPDUNDEOHWKDWVRPDQ\ODQJXDJHPLQRULW\ FKLOGUHQLQWKH86ZRXOGEHFODVVL¿HGDV³QRQYHUEDOLQERWK(QJOLVKDQG WKHLUQDWLYHODQJXDJH´DV3\OH S$ GHVFULEHGZKHQWKH\DUULYHDW VFKRROZKLOHPDMRULW\ODQJXDJHFKLOGUHQDUHQRWVRFODVVL¿HG DQGLQGHHG DUHQRWWHVWHGLQWKLVZD\ $FRQFHSWUHODWHGWRVHPLOLQJXDOLVPLQ&XPPLQV¶VIUDPHZRUNLVWKH GLVWLQFWLRQEHWZHHQ%DVLF,QWHUSHUVRQDO&RPPXQLFDWLRQ6NLOOV %,&6 DQG Journal of Educational Research & Policy Studies &RJQLWLYH$FDGHPLF/DQJXDJH3UR¿FLHQF\ &$/3 &XPPLQV &XPPLQV DOVRXVHVWKHWHUPV³DFDGHPLFODQJXDJH´IRU&$/3 DQG³FRQYHUVDWLRQDOODQJXDJH´IRU%,&6³&RQVLGHUDEO\OHVVNQRZOHGJHRI language itself is usually required to function appropriately in interpersonal FRPPXQLFDWLYHVLWXDWLRQVWKDQLVUHTXLUHGLQDFDGHPLFVLWXDWLRQV´&XPPLQV FODLPVDQGDFDGHPLFODQJXDJHXVXDOO\LQYROYHV³PXFKPRUHORZ frequency vocabulary, complex grammatical structures, and greater demands RQPHPRU\DQDO\VLVDQGRWKHUFRJQLWLYHSURFHVVHV´ S $V0DF6ZDQ DQG5ROVWDG SRLQWRXWWKH%,&6&$/3GLVWLQFWLRQKDVPDQ\RIWKH same negative conceptual