Examining the Construct of Proficiency in a University‟S American Sign Language
Total Page:16
File Type:pdf, Size:1020Kb
Examining the Construct of Proficiency in a University‟s American Sign Language (ASL) Program: A Mixed-Methods Study By Josée-Anna Tanner A thesis submitted to the Faculty of Graduate and Postdoctoral Affairs in partial fulfilment of the requirements for the degree of Master of Arts in Applied Linguistics and Discourse Studies Carleton University Ottawa, Ontario © 2014 Josée-Anna Tanner Abstract American Sign Language (ASL) has become increasingly popular as a second language option at universities and colleges in North America. While a growing number of hearing, adult learners are enrolling in ASL classes, this has not been paralleled (yet) by an equal development in ASL research. There has been insufficient investigation into what constitutes ASL proficiency development and how proficiency can be validly and reliably assessed for this group of learners. This mixed-methods study explores the ASL program at a Canadian university. It investigates the construct of proficiency from three angles: instructors' understanding and definitions of ASL proficiency; how student proficiency is determined through current assessment practices and; student responses to assessment practices. Results of this study suggest that in this context ASL proficiency is not clearly defined. Without a clear construct definition, what current ASL assessments are actually measuring is unknown, and consequently may not be providing valid results. ii Acknowledgements Thank you to my supervisor, Janna Fox. Your guidance and support throughout this process were greatly appreciated, and your experience and insight were invaluable. I am also indebted to those who agreed to volunteer their time and be participants in this study. Thank you for sharing your experiences with me. A very special thank you to Jon Kidd, for helping me get here in the first place. Thank you also to my family, especially my Mum, for cheering for me through this and all of my endeavours. Thank you to Claire for all of the talks about work, life, writing and everything else. And finally thank you to my partner Liam for helping me stay grounded. You have been wonderfully supportive throughout this process – from offering insightful feedback to making sure I stopped working when it was time to stop working. Thank you. iii List of Abbreviations ACTFL – American Council on the Teaching of Foreign Languages ASL – American Sign Language ASLTA – American Sign Language Teachers Association CEFR – Common European Framework of Reference for Languages CODA – Child of Deaf Adults. Hearing child with one (or more) Deaf parents. EFA – Exploratory Factor Analysis LSQ – Langues des signes québécoise FL – Foreign language IELTS – International English Language Testing System L2 – Second language. Any language learned after the first or native language. M2 – Second modality. Any language learned which relies on a different channel of articulation than the first or native language. Usually refers to hearing people learning a signed language. MST – Maritime Sign Language NCATE – National Council for Accreditation of Teacher Education SLA – Second language acquisition TOEFL – Test of English as a Foreign Language TPP – Teacher preparation program iv Glossary Curriculum – The course of study within a program. Deaf – (Capital D) People who self-identify as members of the social community and culture which revolves around deafness and the use of sign language. deaf – (Lowercase d) The physical condition of being unable to hear. Program – A group or selection of courses, identified by a university or other educational institution, as being part or the whole of a path of study. v Table of contents Abstract………………………………...…………………………………………....….. ii Acknowledgements ………………...……….……………………………………….… iii List of Abbreviations…………………………...…………………….……………...… iv Glossary …………………………………………………………………………….....…v List of Tables …………………………………………………………………………. viii List of Appendices ………………………………………………………………...………….. ix Chapter 1: Introduction ………………………………………………………………...……. 1 1.1 The Growth of American Sign Language ……………………………………..…. 2 1.2 What is ASL? ……………………………………………………………………...7 1.3 ASL and Multimodality ………………………………………………….…….. 11 Chapter 2: Literature Review ………………………………………………………………. 15 2.1 Cognitive Processing of ASL ………………………………………………….... 15 2.2 Perspectives on Teaching and Learning ASL ………………………………… 21 2.2.1 Teaching ASL as a Second Language ……………………………….…. 21 2.2.2 Learning ASL as a Second Language ……………………………….… 26 2.3 Defining Second Language Proficiency ……………………………………….. 32 2.4 Assessing Second Language Proficiency ………………………………………. 39 2.5 Validity ……………………………………………………………………….….. 46 Chapter 3: Method ……………………………………………………………………...…… 53 3.1 Research Questions ………………………………………………………….….. 53 3.2 Research Site and Design …………………………………………………...….. 54 3.3 Phase One Data Collection: Qualitative Data ………………………………… 55 3.3.1 Teacher-Participants ……………………………………………….….. 55 3.3.2 Teacher-Participant Demographics ……………………………………… 56 3.3.4 Procedures …………………………………………………………….…. 57 3.4 Phase Two Data Collection: Quantitative Data ………………………………. 58 3.4.1 Participants …………………………………………………………..… 58 3.4.2 Student Demographics ………………………………………………… 58 3.4.3 Instrument ……………………………………………………………... 60 3.4.4 Procedures ………………………………………………………...…… 61 3.5 Phase One Analysis: Qualitative Data ………………………………………… 63 3.6 Phase Two Analysis: Quantitative Data ………………………………………… 65 3.7 Ethical Considerations ………………………………………………………..... 66 Chapter 4: Results and Discussion ……………………………………………………….… 68 4.1 Qualitative Results …………………………………………………………...…. 68 4.1.2 Current Assessment Practices …………………………………………. 69 4.1.2.1 ASL Assessment Format ………………………………….... 69 4.1.2.2 Assessment Content ………………………………………… 71 4.1.2.3 Test Type ………………………………………………...…. 72 vi 4.1.2.4 Teacher-Participant Attitudes Toward Testing ………….….. 73 4.1.3 Descriptions of Proficiency ……………………………………………. 74 4.1.4 Grading ………………………………………………………………... 79 4.1.5 Validity Concerns ………………………………………………….….. 81 4.1.5.1 Under-Representation of the Construct …………………..… 82 4.1.5.2 Language Transfer ……………………………………….…. 82 4.1.5.3 Method Effect …………………………………………….… 84 4.1.6 Teacher-Participant Perceptions of Students ………………………….. 86 4.1.7 Change ……………...………………………………………………….... 89 4.2 Summary of Qualitative Results ……………………………………………….. 91 4.2.1 Addressing Research Question #1 …………………………………….. 92 4.2.2 Addressing Research Question #2 …………………………………….. 92 4.3 Quantitative Results ……………………………………………………………. 93 4.3.1 Why ASL? ………………………………………………………..…… 94 4.3.2 Feedback ………………………………………………………….…… 95 4.3.3 Test Format and Method Effect ……………………………………..… 95 4.3.4 Grading ………………………………………………………………... 98 4.3.5 Proficiency Development ……………………………………………….99 4.3.6 The Next Level ……………………………………………………...…101 4.3.7 Percentage of New Material by Level ……………………………..…..102 4.3.8 Assessment Benefits …………………………………………………...103 4.3.9 Productive and Receptive Testing ……………………………………..104 4.4 Proficiency Section Factor Analysis ………………………………………...….105 4.4.1 Factor 1: ASL Discourse Fluency .…………………………………….105 4.4.2 Factor 2: Basic ASL Communication …………………………………105 4.4.3 Factor 3: Language Use Context ………………………………………107 4.4.4 Factor 4: Comprehension …………………………………………..….107 4.4.5 Factor Analysis Summary ………………………………………….…..108 4.5 Quantitative Results Summary …………………………………………….…. 110 4.5.1 Addressing research question #3 …………………………………..… 111 Chapter 5: Conclusion ……………………………………………………………………... 114 5.1 Limitations ………………………………………………………………….….. 114 5.2 Conclusions About ASL Proficiency Development ………………………….. 115 5.3 Conclusions About ASL Assessment Validity ……………………………….. 117 5.4 Consequences of Current Assessment Practices ……………………….……. 118 5.5 Future directions ………………………………………………………….…… 120 References ……………………………………………………………………………..……. 122 Appendices …………………………………………………………………………......…… 136 vii List of Tables Table 1: Teacher-participants by level of experience and hearing status ………..……. 57 Table 2: ASL enrolment by course/level code, fall 2013 …………………………… 59 Table 3: Cross-tabulation between instructor hearing status and student-reported amount of feedback……………………………………………………………...………. 95 Table 4: Student response by ASL level to questionnaire item #32 ………………… 101 Table 5: Student response by ASL level to questionnaire item #33 ………………… 101 Table 6: Student response by ASL level to questionnaire item #34 ………………… 101 Table 7: Cross-tabulation between ASL level and student-reported amount of new material ……………………………………………………………………….. 102 Table 8: Pattern Matrix for 4-Factor Solution (EFA) …………….…………..…… 106 viii List of Appendices Appendix A: Ethics Certificate …………………………………………………..…… 137 Appendix B: Teacher-Participant Interview Questions ………………………….…… 138 Appendix C: Student Questionnaire …………………………….……………..………139 ix Chapter 1: Introduction I have been an ASL instructor for just two years. I am still a new teacher; I do not have the years of experience that others possess. I am also hearing and my first exposure to ASL was as a second language and a second form of communication. While I have only been involved in teaching ASL for a short time, much of that time has been spent planning, administering, and grading assessments. I arrived on the assessment scene with very little knowledge, but started learning fast. It was as I was teaching my second class that I began to wonder more about ASL assessment, and language testing in general. What was I really assessing? What did it mean for a student to be proficient in ASL at different levels, and what