Academic Bias in Language Testing: a Construct Validity Critique of the IPT I Oral Grades K-6 Spanish Second Edition (IPT Spanish)

Academic Bias in Language Testing: A Construct Validity Critique of the IPT I Oral Grades K-6 Spanish Second Edition (IPT Spanish)

Jeff MacSwan Arizona State University

Kate Mahoney State University of New York

Abstract Construct validity concerns for the IPT I Oral Grades K-6 Spanish Second (GLWLRQ ,376 DVDPHDVXUHRIQDWLYHRUDOODQJXDJHSUR¿FLHQF\DUHH[DPLQHG 7KHH[DPLQDWLRQLQFOXGHGGHVFULELQJDVXEVHWRILWHPVWKDWFRQWULEXWHVPRVW WRRYHUDOOVFRUHDQGQDWLYHODQJXDJHSUR¿FLHQF\GHVLJQDWLRQ&RUUHODWLRQV EHWZHHQWKLVVXEVHWRILWHPVDQGWKHRYHUDOOVFRUHDUHFDOFXODWHG(PSLULFDO evidence suggests that if a student cannot or does not answer four questions from the second section in a complete sentence, then the student will either EHODEHOHGDVDQRQVSHDNHURUDOLPLWHGVSHDNHULQWKHQDWLYHODQJXDJH7KH authors argue that the requirement that speakers respond in complete sentences UHÀHFWVDQDwYHYLHZRIODQJXDJHSUR¿FLHQF\LQFRQVLVWHQWZLWKOLQJXLVWLF research, and characterize the requirement as “academic bias,” that is, a prejudice resulting from a confusion of academic content knowledge related to language arts with actual linguistic ability, having the result of arbitrarily IDYRULQJPHPEHUVRIWKHHGXFDWHGFODVVHV7KHDXWKRUVFRQFOXGHWKDWWKH,376 should not be used for Spanish language assessment with Spanish-speaking FKLOGUHQ

Introduction

A number of teachers, policy makers and educational researchers have accepted, propagated, and even institutionalized the belief that some VFKRRODJHFKLOGUHQKDYHQRODQJXDJH,QWKH/RV$QJHOHV8QL¿HG6FKRRO 87 District, for instance, the Los Angeles Times reported that 6,800 children were FODVVL¿HGDV³QRQQRQV´DQGVDLGWREH³QRQYHUEDOLQERWK(QJOLVKDQGWKHLU QDWLYHODQJXDJH´ 3\OH &KLOGUHQDUHVRFODVVL¿HGDVDUHVXOWRIQDWLYH ODQJXDJHDVVHVVPHQWUHTXLUHGRUUHFRPPHQGHGIRUQRQ(QJOLVKVSHDNHUVLQ PDQ\SDUWVRIWKH86 We begin with an overview of existing state policies for the native ODQJXDJHWHVWLQJRI(QJOLVKODQJXDJHOHDUQHUVLQWKH86DQGWKHQSUHVHQWDQG critique the dominant paradigm in language minority education, the Threshold +\SRWKHVLVDQGDVVRFLDWHG%,&6&$/3GLVWLQFWLRQDFRQFHSWXDOIUDPHZRUN consistent with native oral language testing practices. As a follow-up to 0DF6ZDQDQG5ROVWDG DQLQWHUQDOYDOLGLW\DQDO\VLVRIWKH6SDQLVK YHUVLRQRIWKH,'($3UR¿FLHQF\7HVW ,376 $PRUL 'DOWRQ D ZLGHO\XVHGQDWLYHODQJXDJHDVVHVVPHQWLVSUHVHQWHGWRVKRZWKDWWKH³QRQ´ RU³OLPLWHG´SUR¿FLHQF\UDWLQJPD\IROORZIURPFRQVWUXFWLUUHOHYDQWIDFWRUV – such as knowledge of school and school culture, in the present instance. It is concluded that, in light of the validity concerns and the lack of relevant HPSLULFDOHYLGHQFHIRUWKHH[LVWHQFHRI³QRQQRQV´DVZHOODVWKHSRVVLEOH negative effects of the label itself, educational agencies should abandon the SUDFWLFHRIURXWLQHO\DVVHVVLQJFKLOGUHQ¶VRUDOQDWLYHODQJXDJHSUR¿FLHQF\

Native Language Assessment Policy in U.S. Schools

$QLPSRUWDQWUHVSRQVLELOLW\RIVFKRROVLQWKH86LVWRGHWHUPLQH ZKHWKHURUQRWDFKLOGNQRZV(QJOLVKVXI¿FLHQWO\ZHOOWRVXFFHHGLQDQDOO (QJOLVKLQVWUXFWLRQDOVHWWLQJ,Q/DXY1LFKROV WKH866XSUHPH &RXUWLQWHUSUHWHG7LWOH9,RIWKH&LYLO5LJKWV$FWWRSURKLELWGLVFULPLQDWLRQ against language-minority children by means of schools failing to provide for their special language-related needs. Thus, schools must determine, for any child enrolling in school, whether the child requires special assistance to XQGHUVWDQGVFKRROFRQWHQW6WDWHVWKDWGRQRWKDYHODZVSURKLELWLQJWKHXVH RIODQJXDJHVRWKHUWKDQ(QJOLVKLQVFKRRODUHDEOHWRSURYLGHFKLOGUHQZLWK bilingual instruction, making academic content comprehensible during the WLPHQHHGHGWROHDUQ(QJOLVK $QHYDOXDWLRQRIFKLOGUHQ¶V(QJOLVKDELOLW\LVDSSURSULDWH±LQGHHG imperative – in light of these considerations. However, many states also require or recommend assessment of children’s native language ability, with the result

Fall 2008 / Volume 8, Number 2 88 WKDWQXPHURXVFKLOGUHQDUHLGHQWL¿HGDV³QRQQRQV´±WKDWLVQRQVSHDNHUVRI (QJOLVKDQGQRQVSHDNHUVRIWKHLUKRPHODQJXDJH W\SLFDOO\6SDQLVKLQWKH 86FRQWH[W DFRQGLWLRQKLVWRULFDOO\NQRZQDV³VHPLOLQJXDOLVP´ &XPPLQV 6NXWQDEE.DQJDV 7DEOHOLVWVVWDWHVWKDWUHTXLUHRUUHFRPPHQG QDWLYHODQJXDJHWHVWLQJDQG7DEOHSUHVHQWVWKHPRVWIUHTXHQWO\XVHGWHVWV RIQDWLYHODQJXDJHDELOLW\ DVRIVHH0DKRQH\ 0DF6ZDQ

7DEOH States which require or recommend native language assessment for English /DQJXDJH/HDUQHUV (//V DVSDUWRIWKHLGHQWL¿FDWLRQSURFHVV 0DKRQH\ MacSwan, 2005) and the number of ELLs in each of these states (Kindler, 2002), as of 1999-2000 Total State Percentage of Total State ELL Students Enrollment State Enrollment Arizona &RQQHFWLFXW 'LVWULFWRI&ROXPELD Hawaii Illinois Mississippi Ohio Oklahoma Oregon 6RXWK'DNRWD Texas 9LUJLQ,VODQGV 9LUJLQLD Total 1,043,614 12,482,502

Rethinking the Dominant Paradigm in Language Minority Education

The practice of assessing the native language of bilingual children, LIQRWGLUHFWO\GHULYHGIURPWKH7KUHVKROG+\SRWKHVLVDQG%,&6&$/3 IUDPHZRUN &XPPLQV LVFRQFHSWXDOO\VXSSRUWHGE\ it. The Threshold Hypothesis posits that children may enter into a state of ³GRXEOHVHPLOLQJXDOLVP´EURXJKWDERXWE\DSURFHVVRIQDWLYHODQJXDJH

Journal of Educational Research & Policy Studies 7DEOH Tests Used by States for Native Language Assessment (Mahoney & MacSwan, 2005)

Number of Instrument 6WDWHV

/DQJXDJH$VVHVVPHQW6FDOH6SDQLVK /$6 ,GHD3UR¿FLHQF\7HVW6SDQLVK ,37 :RRGFRFN0XQR]6SDQLVK %LOLQJXDO,QYHQWRU\RI1DWXUDO/DQJXDJH %,1/ %LOLQJXDO6\QWD[0HDVXUH6SDQLVK %60 3HDERG\3LFWXUH9RFDEXODU\7HVW6SDQLVK 3397 subtraction, as a result of contact with a second language in the early years. 6HPLOLQJXDOLVPDOVRNQRZQDV³OLPLWHGELOLQJXDOLVP´ &XPPLQV KDV EHHQGH¿QHGLQYDULRXVZD\VEXWLWLVPRVWFRPPRQO\FRQVLGHUHGWRLQGLFDWH ³ORZOHYHOVLQERWKODQJXDJHV´RU³OHVVWKDQQDWLYHOLNHFRPPDQGRIWKH YRFDEXODU\DQGV\QWDFWLFVWUXFWXUHV´RIHLWKHUODQJXDJH &XPPLQVS &XPPLQV GH¿QHGWKH7KUHVKROG+\SRWKHVLVDVIROORZV Negative cognitive and academic effects are hypothesized to result from low levels of competence in both languages or what 6FDQGLQDYLDQUHVHDUFKHUV HJ+DQVHJDUG>@6NXWQDEE .DQJDV 7RXNRPDD KDYHWHUPHG³VHPLOLQJXDOLVP´RU ³GRXEOHVHPLOLQJXDOLVP´«(VVHQWLDOO\WKHORZHUWKUHVKROG level of bilingual competence proposes that bilingual children’s FRPSHWHQFHLQDODQJXDJHPD\EHVXI¿FLHQWO\ZHDNDVWRLPSDLU the quality of their interaction with the educational environment WKURXJKWKDWODQJXDJH S

Although the Threshold Hypothesis is widely accepted, no relevant HYLGHQFHKDVEHHQSUHVHQWHGWRVXSSRUWWKH³VHPLOLQJXDOLVP´LGHDHPEHGGHG ZLWKLQLW3DXOVWRQ IRULQVWDQFHUHYLHZHGQXPHURXV6FDQGLQDYLDQ studies which sought linguistic evidence for the existence of semilingualism LQ6ZHGHQDQGFRQFOXGHGWKDW³WKHUHLVno empirical evidence to support the H[LVWHQFHRIVXFKDODQJXDJHGHYHORSPHQWKLDWXVDV³>VHPLOLQJXDOLVP@´ S HPSKDVLVLQRULJLQDO 0DF6ZDQ UHYLHZHGUHSXWHGHYLGHQFHIURP

Fall 2008 / Volume 8, Number 2 studies of language variation, linguistic structure, school performance, and language loss, and concluded that all of it was either spurious or irrelevant. 0DF6ZDQ HPSKDVL]HGWKDWVHPLOLQJXDOLVPDVSUHVHQWHGE\LWV proponents, is indistinguishable from classical linguistic prescriptivism. The semilingualism idea contrasts markedly with empirically grounded ZRUNRQFKLOGODQJXDJH$V&KRPVN\ REVHUYHGORQJDJR A consideration of the character of the grammar that is acquired, the degenerate quality and narrowly limited extent of the available data, the striking uniformity of the resulting grammars, and their independence of intelligence, motivation and emotional state, over wide ranges of variation, leave little hope that much of the structure of language can be learned by an organism initially uninformed DVWRLWVJHQHUDOFKDUDFWHU S &KRPVN\¶VVWDWHPHQWSDLQWVDSLFWXUHRIFKLOGUHQDVLQZDUGO\GULYHQODQJXDJH learners who acquire their language perfectly and without instruction. During WKHPRVWDFWLYHDFTXLVLWLRQSHULRG DJHV FKLOGUHQOHDUQDSSUR[LPDWHO\WHQ to twelve new words a day, often on one exposure and in highly ambiguous FLUFXPVWDQFHV *OHLWPDQ /DQGDX &KLOGUHQNQRZWKLQJVDERXW elementary aspects of sentence structure for which they have no evidence at DOO 3LQNHU DQGLQFDVHVRIFUHROL]DWLRQFKLOGUHQDFTXLUHV\QWDFWLFDOO\ and morphologically complex linguistic systems in accordance with universal principles of grammar in the presence of highly degenerate, rudimentary adult ODQJXDJHLQSXW SLGJLQV %LFNHUWRQ In an extensive review of research on child language in the preschool \HDUV7DJHU)OXVEHUJ UHSRUWHGWKDW³E\WKHWLPHFKLOGUHQEHJLQVFKRRO they have acquired most of the morphological and syntactic rules of their ODQJXDJH´DQGSRVVHVVDJUDPPDUHVVHQWLDOO\LQGLVWLQJXLVKDEOHIURPDGXOWV S 7KLVYLHZUHSUHVHQWVDFRQVHQVXVDPRQJUHVHDUFKHUVLQFKLOGODQJXDJH acquisition, where the matter has been investigated empirically for about three GHFDGHV*LYHQWKHVHIDFWVLWLVUHPDUNDEOHWKDWVRPDQ\ODQJXDJHPLQRULW\ FKLOGUHQLQWKH86ZRXOGEHFODVVL¿HGDV³QRQYHUEDOLQERWK(QJOLVKDQG WKHLUQDWLYHODQJXDJH´DV3\OH S$ GHVFULEHGZKHQWKH\DUULYHDW VFKRROZKLOHPDMRULW\ODQJXDJHFKLOGUHQDUHQRWVRFODVVL¿HG DQGLQGHHG DUHQRWWHVWHGLQWKLVZD\ $FRQFHSWUHODWHGWRVHPLOLQJXDOLVPLQ&XPPLQV¶VIUDPHZRUNLVWKH GLVWLQFWLRQEHWZHHQ%DVLF,QWHUSHUVRQDO&RPPXQLFDWLRQ6NLOOV %,&6 DQG

Journal of Educational Research & Policy Studies &RJQLWLYH$FDGHPLF/DQJXDJH3UR¿FLHQF\ &$/3 &XPPLQV &XPPLQV DOVRXVHVWKHWHUPV³DFDGHPLFODQJXDJH´IRU&$/3 DQG³FRQYHUVDWLRQDOODQJXDJH´IRU%,&6³&RQVLGHUDEO\OHVVNQRZOHGJHRI language itself is usually required to function appropriately in interpersonal FRPPXQLFDWLYHVLWXDWLRQVWKDQLVUHTXLUHGLQDFDGHPLFVLWXDWLRQV´&XPPLQV FODLPVDQGDFDGHPLFODQJXDJHXVXDOO\LQYROYHV³PXFKPRUHORZ frequency vocabulary, complex grammatical structures, and greater demands RQPHPRU\DQDO\VLVDQGRWKHUFRJQLWLYHSURFHVVHV´ S $V0DF6ZDQ DQG5ROVWDG SRLQWRXWWKH%,&6&$/3GLVWLQFWLRQKDVPDQ\RIWKH same negative conceptual consequences as does the Threshold Hypothesis. $VZLWKVHPLOLQJXDOLVPDSUREOHPHPHUJHVEHFDXVH%,&6&$/3HTXDWHV the language of school, and hence the language of the educated classes, with language that is inherently more complex, richer, and which places greater GHPDQGVRQFRJQLWLYHUHVRXUFHV6HH0DF6ZDQDQG5ROVWDG IRUIXUWKHU discussion. 2QHNLQGRISXUSRUWHGHYLGHQFHIRUVHPLOLQJXDOLVPDQGWKH%,&6 &$/3GLVWLQFWLRQFRPHVIURPVWXGHQWV¶WHVWUHVXOWVRQRUDOQDWLYHODQJXDJH assessments such as the Language Assessment Scales Oral-Español /$62 (VSDxRO WKH,GHD3UR¿FLHQF\7HVW6SDQLVK ,736 DQGWKHWoodcock-Muñoz, ZKLFKSXUSRUWVWRWHVWDFKLOG¶VOHYHORI&$/3LQ6SDQLVK,QWKHSUHVHQWSDSHU ZHOLPLWRXUDQDO\VLVWRFRQVWUXFWYDOLGLW\LVVXHVLQWKH,3766HH0DF6ZDQ 5ROVWDGDQG*ODVV IRUDQDQDO\VLVRIWKH3UH/$62UDO(VSDxRODQG 0DF6ZDQDQG5ROVWDG IRUDQH[WHUQDOYDOLGLW\DQDO\VLVRIWKH/$62 (VSDxRODQGWKH,376

&RQVWUXFW9DOLGLW\RIWKH,GHD3UR¿FLHQF\7HVW6SDQLVK ,376 &RQVWUXFWYDOLGLW\LVWKHGHJUHHWRZKLFKDWHVWDFWXDOO\PHDVXUHVWKH construct of interest – oral language, in the case at hand. In attempting to UH¿QHWKHQRWLRQRIYDOLGLW\0HVVLFN QRWHG³9DOLGLW\LVDQLQWHJUDWHG evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions EDVHGRQWHVWVFRUHVRURWKHUPRGHVRIDVVHVVPHQW´ S 7KH,376SXUSRUWVWREHDWHVWRIODQJXDJHDELOLW\DQGSXUSRUWVWKDWLWV VFRUHVLQGLFDWHWKHGHJUHHWRZKLFKDFKLOGKDVDFTXLUHGDODQJXDJH±6SDQLVK LQWKHFDVHRIWKH,376,QWKHFDVHRIDVVHVVLQJ6SDQLVKDVDVHFRQGODQJXDJH we would naturally expect a wide range of ability levels. But it should come

Fall 2008 / Volume 8, Number 2 DVDVXUSULVHWKDWFKLOGUHQZKRDUHERUQDQGUDLVHGLQDPRQROLQJXDO6SDQLVK speaking environment would be assessed as non- or limited speakers of 6SDQLVK To assist in determining whether the scores of language tests such as the ,376DUHYDOLGLWZLOOEHXVHIXOWRHYDOXDWHWKHLPSOLFLW³WKHRUHWLFDOUDWLRQDOH´ LQ0HVVLFN¶VWHUPVRIWKH,3767KHEDVLFTXHVWLRQXQGHUFRQVLGHUDWLRQLV WKLV'RPRUHFRUUHFWUHVSRQVHVRQWKHWHVWLWHPVZKLFKFRPSULVHWKH,376 consistently provide evidence of higher 6SDQLVKRUDOODQJXDJHSUR¿FLHQF\? ,IVRWKHQWKHWHVWZLOOFRUUHFWO\PHDVXUHWKHLQWHQGHGFRQVWUXFW6SDQLVK RUDOODQJXDJHSUR¿FLHQF\+RZHYHUWKHWHVWZLOOODFNFRQVWUXFWYDOLGLW\WR the extent that correctly answering items requires knowledge of some other, irrelevant domain – say, for instance, knowledge of school culture and school subjects. $WUDGLWLRQDOLGHDLQODQJXDJHDUWVIUHTXHQWO\DWWULEXWHGWR3ODWRLVWKDW D³FRPSOHWHVHQWHQFH´FRQVLVWVRIDVXEMHFWDQGDSUHGLFDWHDQGWKDW³FRUUHFW ODQJXDJH´RUDWOHDVW³EHWWHUVSHHFK´LVFKDUDFWHUL]HGE\VXFKXWWHUDQFHV,QWKH real world, however – and even in academia – people communicate in phrases shorter than sentences with very high frequency. Although no arguments have EHHQDGYDQFHGWRVXVWDLQWKHLGHDWKDWVSHHFKFKDUDFWHUL]HGE\³FRPSOHWH VHQWHQFHV´LVEHWWHUVSHHFKWKHQRWLRQKDVIRXQGLWVZD\LQWRFRQVLGHUDEOH HGXFDWLRQDOZRUNLQPLQRULW\ODQJXDJHLVVXHV)RUH[DPSOH6FKDW]PDQQDQG 6WUDXVV ZKRKDGLQWHUYLHZHGPRQROLQJXDOPHPEHUVRIWKHORZHUDQG middle classes about their impressions after the occurrence of a disaster, found that the former used lots of emotional language which reputedly gave rise to ³HOOLSWLFDOV\QWD[´DQGOLPLWHGWKHPWRFRQYH\LQJPHDQLQJ³LPSOLFLWO\´ZKLOH WKHHGXFDWHGFODVVHVFRXOGGRVR³H[SOLFLWO\´ ,QDFODVVLFFULWLTXHRIVXFKDSSURDFKHV/DERY UHYLHZHGZRUN E\%HUHLWHUDQGFROOHDJXHV HJ%HUHLWHU (QJHOPDQQ UHJDUGLQJWKH UHODWLRQVKLSEHWZHHQ$IULFDQ$PHULFDQ9HUQDFXODU(QJOLVK $$9( DQGWKH poor educational achievement of African-American school children. Bereiter DQG(QJHOPDQQUHSRUWHGWKDWWKHIRXU\HDUROGVWKH\VWXGLHGFRPPXQLFDWHG E\JHVWXUHV³VLQJOHZRUGV´DQG³DVHULHVRIEDGO\FRQQHFWHGZRUGVRU SKUDVHV´$FFRUGLQJWR%HUHLWHUDQG(QJHOPDQQWKHVHFKLOGUHQFRXOG³ZLWKRXW H[DJJHUDWLRQ«PDNHQRVWDWHPHQWVRIDQ\NLQG´DQGFRXOGQRWDVNTXHVWLRQV 2ISDUWLFXODUVLJQL¿FDQFHZDV%HUHLWHUDQG(QJHOPDQQ¶VH[SHFWDWLRQWKDW FKLOGUHQDQVZHULQFRPSOHWHVHQWHQFHV,QUHVSRQVHWRWKHTXHVWLRQ³:KHUH

Journal of Educational Research & Policy Studies LVWKHVTXLUUHO"´%HUHLWHUDQG(QJHOPDQQ¶VVXEMHFWVWHQGHGWRDQVZHU³,Q WKHWUHH´±DUHVSRQVHWKHUHVHDUFKHUVFKDUDFWHUL]HGDVLOORJLFDODQGEDGO\ IRUPHG$V/DERY SRLQWHGRXWWKHUHVSRQVH³,QWKHWUHH´LVWKHQDWXUDO response in this context, and the one that anybody would use under normal circumstances–except, perhaps, in the context of an academic exercise. ,QGHHGDQH[DPLQDWLRQRIWKHFRQWHQWVRIWKH,376VXJJHVWVWKDWWKH test is much more an assessment of academic knowledge than of language DELOLW\)RULQVWDQFHWKHVHFRQGSDUWRIWKH,376DVNVIRXUTXHVWLRQVWRZKLFK students are required to provide answers in complete sentences, as shown in 7DEOH6WXGHQWVZKRPLVVWKHVH RUDQ\RWKHU IRXUTXHVWLRQVRQWKLVSDUW DUHODEHOHG³OLPLWHG6SDQLVKVSHDNLQJ´$IWHUD¿UVWLQFRUUHFWUHVSRQVHWKH WHVWDGPLQLVWUDWRUGLUHFWVWKHVWXGHQWWR³DQVZHUWKHTXHVWLRQLQDFRPSOHWH VHQWHQFH´

7DEOH Items on “Level C” of the ITP Spanish that Require Students to Answer in Complete Sentences Item 5HTXLUHGVWXGHQWUHVSRQVH 3URPSW ¢4XpHVWiKDFLHQGRHOQLxR" (O QLxR HVWiOH\HQGR 3LFWXUHRIER\ >:KDWLVWKHER\GRLQJ"@ estudiando. looking at >7KHER\LVUHDGLQJ book. VWXG\LQJ@ ¢&XiQWRVPDQRVWHQJR\R" 8VWHGWLHQHGRVPDQRV7~ None. >+RZPDQ\KDQGVGR,KDYH"@ WLHQHV« >&DQKRUVHVUXQ"@ >1RHOHSKDQWVGRQ¶WÀ\@ >'RHOHSKDQWVÀ\OLNHELUGV"@

3XWWLQJDVLGHWKHLQFRQVLVWHQF\LQWKHLPSOLFLWGH¿QLWLRQRID³FRPSOHWH VHQWHQFH´DQGWKHKHDYLO\GHFRQWH[WXDOL]HGQDWXUHRIWKHLWHPVZHPXVW ask whether the ability to recognize or produce a complete sentence on demand ought to factor into our assessment of a native speaker’s knowledge of language. Indeed, few of us would produce answers like those required DERYHLIDVNHGWKHVHTXHVWLRQV7KHQDWXUDOUHVSRQVHWRLWHPLQ7DEOHIRU instance, is simply Leyendo or Estudiando ³5HDGLQJ6WXG\LQJ´

Fall 2008 / Volume 8, Number 2 Indeed, one’s ability to answer in a fragment reveals detailed covert knowledge of linguistic structure. For instance, if asked a question such as item LQ7DEOHZHUHO\RQRXUNQRZOHGJHRIWKHLQWHUQDOVWUXFWXUHRIWKHSKUDVH to determine possible shortened forms of the sentence, such as No and No, they don’tZHFDQDOVRUHÀHFWRQRXUNQRZOHGJHRIODQJXDJHWRGHWHUPLQHZKLFK VKRUWHQHGYHUVLRQVDUHQRWVWUXFWXUDOO\SRVVLEOH HJ1RWKH\No, they do In fact, answering the question in the desired way requires that we suspend our knowledge of pragmatics, which tells us that we can delete recoverable information, in order to comply with an institutional requirement to respond in a so-called complete sentence. Most people learn about the concept of complete sentences in school, which is why the ability to identify them so they can be produced on demand should be regarded as part of the domain of academic achievement, not an aspect of knowledge of language. A language is a set of expressions generated E\DJUDPPDUZKLFKPDSVPHDQLQJWRVRXQG &KRPVN\ 9HU\ early on, children exhibit complex knowledge of word order, word structure, pronunciation, discourse structure, and appropriate use of language in distinct VLWXDWLRQV$OOQRUPDOFKLOGUHQH[KLELWWKLVNQRZOHGJHUHJDUGOHVVRIWKHLUVSHFL¿F cultural background or life experiences. By contrast, knowledge of particular communities and cultural practices – including those internal to the school ±GHSHQGXSRQRQH¶VLQWHUHVWVDQGVSHFL¿FODQJXDJHOHDUQLQJHQYLURQPHQWV ,IZHGH¿QHODQJXDJHSUR¿FLHQF\ in such a way as to include highly particular cultural knowledge, what should be regarded as a simple cultural difference suddenly becomes a linguistic dividing line which enormously privileges those with more socially valued cultural capital in hand. Only a small segment of WKHKXPDQUDFHH[SHULHQFHVIRUPDOVFKRROLQJDQGHYHQIHZHUH[FHODWLWEXW all of us know a language. It is therefore necessary to examine empirically the GHJUHHWRZKLFKWKH³FRPSOHWHVHQWHQFH´UHTXLUHPHQWRIWKH,376LQÀXHQFHV RYHUDOOSUR¿FLHQF\GHVLJQDWLRQV

Method

Participants 3DUWLFLSDQWVLQFOXGHG¿UVWWKURXJKWKLUGJUDGH+LVSDQLFVWXGHQWV LQWKH3KRHQL[0HWURSROLWDQDUHD7KLVVDPSOHFDPHIURPWZRVFKRROVDQG FRQVLVWHGRI ¿UVWJUDGHUV VHFRQGJUDGHUV

Journal of Educational Research & Policy Studies WKLUGJUDGHUVDQGRQH IRXUWKJUDGHU7ZRVWXGHQWVDUHPLVVLQJJUDGH level information. The sample is distributed approximately evenly among boys DQGJLUOV$OOSDUWLFLSDQWVZHUHVHOHFWHGIURP6SDQLVKVSHDNLQJKRXVHKROGV

Instrument $VPHQWLRQHGWKH,376LQWHQGVWRGHWHUPLQHWKHOHYHORI6SDQLVKRUDO ODQJXDJHSUR¿FLHQF\RI+LVSDQLFSULPDU\VFKRROVWXGHQWVDQGRWKHUV,WDOVR provides the information necessary to immediately designate these students DV1RQ)OXHQW 166 /LPLWHG /66 RU)OXHQW6SDQLVK6SHDNLQJ )66 DV well as provide diagnostic information for instruction. Test authors claim WKH,376DVVHVVHVIRXUEDVLFDUHDVRI6SDQLVKRUDOODQJXDJHSUR¿FLHQF\ 9RFDEXODU\&RPSUHKHQVLRQ6\QWD[DQG9HUEDO([SUHVVLRQZKLFKLQFOXGHV DUWLFXODWLRQ7KHUHDUHVL[OHYHOVRIGLI¿FXOWLHVWHVWHG/HYHOV$%&'( and F. All students are tested individually.

Test Format 7KHUHDUHVL[VHFWLRQV OHYHO$%&'() DQGVWXGHQWVFRQWLQXH WKURXJKWKHWHVWXQWLOWKH\PLVVDVSHFL¿HGQXPEHURILWHPV6HFWLRQ%KDV TXHVWLRQVVHFWLRQ&KDVTXHVWLRQVVHFWLRQ'KDVTXHVWLRQVVHFWLRQ( KDVTXHVWLRQVDQGVHFWLRQ)KDVTXHVWLRQV$FFRUGLQJWRWKHWHFKQLFDO PDQXDOVWXGHQWVDUHOHGWKURXJKSURJUHVVLYHO\PRUHGLI¿FXOWLWHPVLQOH[LFRQ grammar, and comprehension.

Procedure Data were collected in two different elementary schools in a large urban GLVWULFW3DUWLFLSDWLRQLQWKLVVWXG\ZDVYROXQWDU\ZLWKSHUPLVVLRQJUDQWHGE\ SDUHQWV7KHSDUWLFLSDQWVVHOHFWHGZHUHLGHQWL¿HGDV VFRULQJQRQSUR¿FLHQW LQRUDOODQJXDJHRQWKH/DQJXDJH$VVHVVPHQW6FDOH(QJOLVK /$6(QJOLVK KDYLQJ(//FODVVL¿FDWLRQVWDWXVDQG EHLQJEHWZHHQWKHDJHVRIVL[DQG HLJKW LQFOXVLYH 7UDLQHGQDWLYH6SDQLVKVSHDNLQJSHUVRQQHODGPLQLVWHUHGWKH ,376RQHWLPHIRUHDFKSDUWLFLSDQW7KHDGPLQLVWUDWRURIWKHWHVWFDOFXODWHG WKHVFRUHDQGÀXHQF\OHYHOIRUHDFKSDUWLFLSDQWIROORZLQJWKH,376VFRULQJ instructions.

Fall 2008 / Volume 8, Number 2 Results

Frequencies 7HVWVFRUHVVKRZWKDWRQO\ RIWKHVDPSOHZHUHODEHOHGDV)66 )OXHQW6SHDNHUVRI6SDQLVK )LJXUHVKRZVWKHIUHTXHQFLHVRISUR¿FLHQF\ labels for our sample as determined by the test.

160 135 (77.6%) 140 120 100

80 60 22 17 40 (12.6) (9.8%) Number of Students 20

0 Fluent Limited Non Spanish Spanish Spanish Speaker Speaker Speaker

Proficiency Labels

)LJXUHFrequency of Non-Spanish Speaker, Limited Spanish Speaker, and Fluent Spanish Speaker on the IPT Spanish (N = 174)

Error Pattern Analysis and Item Content Analysis 6WXGHQWVHQGWKH,376RQFHWKH\KDYHPLVVHGDVSHFL¿HGQXPEHURI LWHPVWKHQXPEHURIDOORZDEOHLQFRUUHFWLWHPVLVGLIIHUHQWIRUHDFKVHFWLRQ,W LVDVVXPHGWKDWRQFHVWXGHQWVVFRUHDWWKHFHLOLQJOHYHOVSHFL¿HGIRUDQ\JLYHQ section, they would have answered incorrectly for the remainder of the test. An error pattern analysis for this sample shows most subjects ended WKHWHVWLQVHFWLRQ&,QWKHLWHPOHYHOGDWDEHWZHHQLWHPDQGLWHPWKH VDPSOHVL]HGHFUHDVHGIURPVWXGHQWVWRVWXGHQWV,WHPLVWKHODVW LWHPLQVHFWLRQ&DQGLWHPLVWKH¿UVWLWHPRIVHFWLRQ''HSHQGLQJRQD child’s current grade and testing purpose, failing to advance to section D will UHVXOWLQD/66 /LPLWHG6SDQLVK6SHDNHU GHVLJQDWLRQ

Journal of Educational Research & Policy Studies The dramatic drop in the number of students advancing to section D ZDUUDQWVDFORVHUH[DPLQDWLRQRIVHFWLRQ&DQGDQLWHPFRQWHQWDQDO\VLV $FFRUGLQJWRWHVWGLUHFWLRQVDVWXGHQWZRXOGVWRSDWWKHHQGRIVHFWLRQ&LI KHKDVDWOHDVWIRXULQFRUUHFWDQVZHUVRIWRWDOLWHPV2XULWHPOHYHOFRQWHQW H[DPLQDWLRQXQFRYHUHGH[DFWO\IRXULWHPVLQVHFWLRQ&UHTXLULQJWKHFKLOGWR DQVZHULQFRPSOHWHVHQWHQFHVVKRZQHDUOLHULQ7DEOH

Correlations The four items requiring that a child answer the question in a complete VHQWHQFH DVLQGLYLGXDOLWHPVDQGDVDVXEJURXSZHUHH[DPLQHG 7DEOHVKRZVWKHFRUUHODWLRQVEHWZHHQWKHVHIRXULWHPVWKHLUVXPDQGWKH RYHUDOOSUR¿FLHQF\OHYHO

7DEOH Correlation for the Four Items (17, 18, 21, 22) and Their Sum to the Overall 3UR¿FLHQF\/HYHO 1 Overall 6XPRI$OO Items 3UR¿FLHQF\ Four Level

- 6XPRI$OO Four - Overall 3UR¿FLHQF\ Level -

Note$OO&RUUHODWLRQVDUHVLJQL¿FDQWDWWKHOHYHO.

If a student does not have the overt academic knowledge required to identify a complete sentence, then she is likely to answer incorrectly to items DQGLQVHFWLRQ&,IDFKLOGPLVVHVDOOIRXULWHPVVKHKDVUHDFKHG

Fall 2008 / Volume 8, Number 2 WKHPD[LPXPDOORZDEOHHUURUVLQVHFWLRQ&DQGZLOOKDYHWRVWRSWKHUHE\ DFKLHYLQJDVFRUHGHVLJQDWLQJKHUDV/LPLWHG6SDQLVK6SHDNHU$VRQHVHHVLQ 7DEOHHDFKLQGLYLGXDOLWHPKDVDKLJKFRUUHODWLRQZLWKWKHRYHUDOOSUR¿FLHQF\ level, and the sum of these four items has an especially high correlation with WKHRYHUDOOSUR¿FLHQF\VFRUH p

Frequency of Errors in Four Items 6LQFHLWHPVDQGKDYHDKLJKFRUUHODWLRQZLWKWKHRYHUDOO score, and since the format of the test directs the administrator to stop at WKHHQGRIVHFWLRQ&IRUVWXGHQWVZLWKIRXURUPRUHHUURUVLWLVLPSRUWDQWWR examine these four items and compute error frequencies for our sample, as VKRZQLQ7DEOH

7DEOH Total number of errors from items 17, 18, 21 and 22 1XPEHURI(UURUV 1XPEHURI6WXGHQWV 3HUFHQW7RWDO6WXGHQWV 0

7KHDYHUDJHHUURUUDWHIRUWKHVHIRXULWHPVFRPELQHGLVZLWKD VWDQGDUGGHYLDWLRQRI7DEOHVKRZVWKHQXPEHURIVWXGHQWVZKRPLVVHG two, three, or four complete-sentence items out of a possible four misses to EH ,WLVOLNHO\WKDWRYHUKDOIRIRXUVDPSOHZDVWHUPLQDWHGDWWKH HQGRIVHFWLRQ&EDVHGXSRQWKHLUDQVZHUVWRWKHVHIRXUTXHVWLRQVUHTXLULQJ a complete-sentence answer.

Discussion and Conclusions

:KHWKHUWKH,376WHVWGHVLJQDWHVDVWXGHQWDVDOLPLWHGVSHDNHUUHVWV largely on whether the student has knowledge of what a complete sentence is DQGZKHQSURPSWHGFDQUHVSRQGWRHDFKRIIRXUTXHVWLRQVLQVHFWLRQ&LQD IXOOVHQWHQFH,376LVQRWVRPXFKDPHDVXUHRIRUDOODQJXDJHSUR¿FLHQF\DVLW

Journal of Educational Research & Policy Studies is a measure of school cultural knowledge related to subjects such as language arts, realized in a student’s demonstrated ability to recognize and produce DFRPSOHWHVHQWHQFHRQGHPDQG7KHWHVWLVUHPLQLVFHQWRI/DERY¶V UHEXNHRI%HUHLWHUORQJDJRDQGMXVWL¿HVTXRWDWLRQRIKLVVFROGLQJZRUGV ³7KDWHGXFDWLRQDOSV\FKRORJ\VKRXOGEHVWURQJO\LQÀXHQFHGE\DWKHRU\VR IDOVHWRWKHIDFWVRIODQJXDJHLVXQIRUWXQDWHEXWWKDWFKLOGUHQVKRXOGEHWKH YLFWLPVRIWKLVLJQRUDQFHLVLQWROHUDEOH´ S As researchers and policymakers achieve greater clarity about language SUR¿FLHQF\DQGLWVUHODWLRQVKLSWRDFDGHPLFDFKLHYHPHQWRQHKRSHVWKDW explanations of failure in school will no longer be rooted in children’s linguistic resources at home. At a more practical level, we recommend that VFKRROVGRQRWURXWLQHO\XVHWKH,376RUUHODWHGWHVWVWRDVVHVVFKLOGUHQ¶V RUDOQDWLYHODQJXDJHSUR¿FLHQF\ 6SDQLVKVSHDNLQJFKLOGUHQOLNHDQ\RWKHUV should be referred for assessment for language disorders on a case by case basis, when an informed clinician determines that the child’s language exhibits characteristics typical of children with language impairments. To determine ZKHWKHUFKLOGUHQDUHLQGHHGSUR¿FLHQWLQ6SDQLVKUDWKHUWKDQ(QJOLVKDQG whether they are therefore better served by a bilingual program, much more XVHIXODQGPHDQLQJIXOLQIRUPDWLRQFDQEHREWDLQHGWKURXJKVXI¿FLHQWO\ detailed interviews with a parent and the child.

7KLVUHVHDUFKZDVVXSSRUWHGE\D6PDOO*UDQWIURPWKH6SHQFHU Foundation.

References

$PRUL% 'DOWRQ() Technical manual: IDEA Oral Lan- JXDJH3UR¿FLHQF\7HVW6SDQLVK ,37,2UDO*UDGHV. ndHG %UHD &$%DOODUG 7LJKH3XEOLVKHUV %HUHLWHU& (QJOHPDQQ6 Teaching disadvantaged children in the pre-school(QJOHZRRG&OLIIV1<3UHQWLFH+DOO %LFNHUWRQ' 7KHURRWVRIODQJXDJH$QQ$UERU.DURPD3UHVV &KRPVN\1 $VSHFWVRIWKHWKHRU\RIV\QWD[&DPEULGJH0,7 3UHVV &KRPVN\1 Knowledge of language: Its nature, origin, and use. 1HZ

Fall 2008 / Volume 8, Number 2 &KRPVN\1 7KUHHIDFWRUVLQODQJXDJHGHVLJQLinguistic Inquiry, 36 &XPPLQV- 7KHLQÀXHQFHRIELOLQJXDOLVPRQFRJQLWLYHJURZWK$ V\QWKHVLVRIUHVHDUFK¿QGLQJVDQGH[SODQDWRU\K\SRWKHVHVWorking Pa- pers on Bilingualism, 9 &XPPLQV- /LQJXLVWLFLQWHUGHSHQGHQFHDQGWKHHGXFDWLRQDOGHYHO- opment of bilingual children. Review of Educational Research, 49 &XPPLQV- 7KHUROHRISULPDU\ODQJXDJHGHYHORSPHQWLQSURPRW- LQJHGXFDWLRQDOVXFFHVVIRUODQJXDJHPLQRULW\VWXGHQWV,Q&DOLIRUQLD 6WDWH'HSDUWPHQWRI(GXFDWLRQ (G Schooling and language minority students: A theoretical framework SS /RV$QJHOHV(YDOXDWLRQ 'LVVHPLQDWLRQDQG$VVHVVPHQW&HQWHU&DOLIRUQLD6WDWH8QLYHUVLW\ &XPPLQV- QGHG 6HPLOLQJXDOLVP,QEncyclopedia of language and linguistics SS 2[IRUG(OVHYLHU6FLHQFH/WG &XPPLQV- Language, power and pedagogy: Bilingual children in WKHFURVV¿UH&OHYHGRQ8.0XOWLOLQJXDO0DWWHUV *OHLWPDQ/ /DQGDX% 7KHDFTXLVLWLRQRIWKHOH[LFRQ&DP- EULGJH0,73UHVV /DERY: 7KHORJLFRIQRQVWDQGDUG(QJOLVK,Q):LOOLDPV (G Language and poverty SS &KLFDJR5DQG0F1DOO\ /DXY1LFKROV 868QLWHG6WDWHV6XSUHPH&RXUW86 0DF6ZDQ- 7KH7KUHVKROG+\SRWKHVLVVHPLOLQJXDOLVPDQGRWKHU FRQWULEXWLRQVWRDGH¿FLWYLHZRIOLQJXLVWLFPLQRULWLHVHispanic Journal of Behavioral Sciences, 22 0DF6ZDQ- 5ROVWDG. /LQJXLVWLFGLYHUVLW\VFKRROLQJDQGVR- FLDOFODVV5HWKLQNLQJRXUFRQFHSWLRQRIODQJXDJHSUR¿FLHQF\LQODQJXDJH PLQRULW\HGXFDWLRQ,Q&%3DXOVWRQ 57XFNHU (GV Sociolinguis- tics: The essential readings SS 2[IRUG%ODFNZHOO 0DF6ZDQ- 5ROVWDG. +RZODQJXDJHWHVWVPLVOHDGXVDERXW ODQJXDJHDELOLW\,PSOLFDWLRQVIRUVSHFLDOHGXFDWLRQSODFHPHQWVTeachers College Record, 108 ± 0DF6ZDQ-5ROVWDG. *ODVV*9 'RVRPHVFKRRODJHFKLO- GUHQKDYHQRODQJXDJH"6RPHSUREOHPVRIFRQVWUXFWYDOLGLW\LQWKH3UH /$6(VSDxROBilingual Research Journal, 26

Journal of Educational Research & Policy Studies 0DKRQH\.6 0DF6ZDQ- 5HH[DPLQLQJLGHQWL¿FDWLRQDQG UHFODVVL¿FDWLRQRI(QJOLVK/DQJXDJH/HDUQHUV$FULWLFDOGLVFXVVLRQRI select state practices. Bilingual Research Journal, 29 0HVVLFN6 9DOLGLW\RIWHVWLQWHUSUHWDWLRQDQGXVHResearch Report 553ULQFHWRQ1-(GXFDWLRQDO7HVWLQJ6HUYLFH 3DXOVWRQ&% 6ZHGLVKUHVHDUFKDQGGHEDWHDERXWELOLQJXDOLVP 6WRFNKROP1DWLRQDO6ZHGLVK%RDUGRI(GXFDWLRQ 3LQNHU6 7KHODQJXDJHLQVWLQFW+RZWKHPLQGFUHDWHVODQJXDJHV 1HZ

Fall 2008 / Volume 8, Number 2