Using Vignettes to Build and Assess Teacher Understanding of Instructional Strategies
Total Page:16
File Type:pdf, Size:1020Kb
Using Vignettes To Build and Assess Teacher Understanding of Instructional Strategies Carolyn Jeffries Dale W. Maeder California State University, Northridge University of California, Los Angeles Abstract In the last fifty years, the use of stories in education has included vignettes as an effective stimulus for discussion of real-life contexts and problems. However, vignettes have rarely been used as an assessment tool and there is no reported consensus on their definition and design. This article documents the use of vignettes as an effective method of assessing pedagogical understanding in our teacher development courses from 1995-2003, suggesting that vignettes are significantly correlated with more traditional forms of assessment, are highly predictive of course-ending project performances, and represent an episode of learning in their own right. Finally, we propose a more concise definition and a more rigorous course of study for vignette development and implementation. A critical theme of institutions of higher contexts and problems. education in the preparation and continuing Vignettes have rarely been reported as a education of teachers is that educational research means of assessing mastery of course content. informs and is informed by practice. Identifying Furthermore, there are few definitions or con- and assessing what and how teachers learn during struction criteria for vignettes in these contexts, their professional development is an important and certainly no reported consensus on their component in assisting them to better teach their definition and design. Given the benefits of using students. The use of stories in various formats has vignettes in research, modeling, discussion, and long been a powerful and successful method for reflection, and the need to determine teacher modeling, teaching, and researching behavior and qualification, it is important to determine whether understanding in general education, health sci- vignettes can also be an effective assessment tool ences, social sciences, and behavioral sciences. in teacher education. One type of story, the vignette, has been used An important step in this process is the collec- in the social sciences and more recently in the tion of evidence that vignettes are a reliable as- health and education fields as a reflective, re- sessment tool that complements more traditional search, and modeling tool (Callicott, 2003; Camp- assessment styles. In addition, it should be deter- bell, n.d.; Chambers, 1999; Galguera, 1998; Hughes mined whether vignettes can augment the infor- & Huby, 2002; King, Murray, Salomon, & Tandon, mation gathered in more traditional assessments. 2002a; Kruse, 1997; New Hampshire Equity Hand- Besides content and skill mastery, do vignettes book Writing Team, n.d.; Pransky & Bailey, 2002; help to measure affective goals, such as the moti- Schwartz & Riedesel, 1994; TESOL, 2001; Vaughn vation to learn? Do vignette assessments provide & Klingner, 1999; Volkmann, 1998). Researchers evidence of problem solving and critical thinking? and educators have found vignettes to be very Furthermore, it is critical to determine if effective in these contexts for several reasons: vignettes can assist in state-mandated assessment vignettes are relatively easy to construct, they of teacher understanding of pedagogy. The chal- provide a useful focus and stimulus for discus- lenge of assessing portfolios may be partially sion, they are valuable in addressing difficult-to- addressed through vignette scoring guides that explore and sensitive topics, they can be used with quantify responses that are unique and authentic. individuals and groups, and they reflect real-life Finally, vignettes should be studied for evidence Volume XXVII • Number 1 & 2 • Fall 2004 & Spring 2005 17 Carolyn Jeffries & Dale W. Maeder of an assessment as an episode of learning. Does opment Oriented Research in Agriculture, n.d.; an assessment task that encourages modeling, Pesonen, n.d.), and vignettes. rehearsal, and transfer provide more opportuni- ties for learning than more traditional forms of Vignettes in Education and Research assessment, (e.g., multiple-choice, fill-in items)? A recent review of the literature on vignettes Since 1995, we have been using vignettes as a revealed a wide range of definitions, usage, con- method of assessing pedagogical understanding struction, testing, and examples. Definitions were in teacher development courses. We believe the rarely provided and varied greatly, which caused vignette to be a highly effective assessment tool confusion of vignettes with other types of stories. and suspect that vignettes have a strong predic- In fact, many studies and articles used the terms tive component as well as provide a rich learning case study, case story, scenario, and vignette experience for eliciting and promoting teacher interchangeably, incorrectly, or uniquely, (e.g., understanding. We would like to share our experi- Jackson, 1998; Kruse, 1997; Walen & Hirstein, ences and data with teacher educators and re- 1995; Weigle & Scotti, 2000; Weiner, Rand, Paga- searchers in order to propose a course of study on no, Obi, Hall, & Bloom, 2001). the use of vignettes in assessing pedagogical Broadly defined, vignettes are “short stories understanding in teacher education programs. about hypothetical characters in specific circum- Based on the literature and our experiences, we stances to whose situation the interviewee is would also like to propose a more concise defini- invited to respond…moving from the abstract to tion and establish more rigorous criteria for context-specific” (Finch, 1987, p. 106). Vignettes vignette development and implementation in this consist of text, images, or other forms of stimuli, assessment process. ranging from short written prompts to live events, to which research participants are asked to re- Review of Literature spond (Hughes & Huby, 2002). Herman (1998) Stories as Educational Tools uses the term case vignette to describe a written A story is defined by Cambridge Advanced description, photograph, or videotaped scene as a Learner’s Dictionary (Cambridge University Press, brief glimpse of an educational situation. Camp- 2003) as “a description, either true or imagined, of bell (n.d.) comes closest to an operational defini- a connected series of events and, often, the charac- tion: ters involved in them.” Stories have been used in A vignette is a short story without an ending. many different forms and formats as a powerful It is short, but not too short to present an and successful method for modeling, teaching, issue. It is detailed, but not so detailed that the and researching behavior and understanding in underlying issue gets lost. A vignette presents general education, health sciences, social sciences, an issue, such as the under-representation of and behavioral sciences. Story formats include case girls in advanced math courses, in a context studies, long narrative descriptions of real or with which individuals can identify. A good hypothetical situations in which learners are vignette has fewer complexities and personali- asked to identify or solve a problem (Gideonse, ties than real life, sets up a situation in which 1999; Hrabe, Kinzie, & Julian, 2001; Jackson, 1998; there is no one “right” answer, and is flexible Loughner, Harvey, & Mil-heim, 2001; Marsick, enough that individuals from different groups 1998); case stories, stories that simulate the real (teacher/administrator, female/male, liberal/ world but are written by individuals within the conservative) can identify with the story and classroom and told from their perspectives (Hunt- bring their perspective forward in discussions er & Hatton, 1998; Maslin-Ostrowski & Ackerman, of solutions. (p. 2) 1998); scenarios, narrative descriptions that contain a set of realistic assumptions and facts about the Vignette Construction and Design future used to provide a unified context for In addition to a lack of consensus on vignette decision-making (International Centre for Devel- definition, there was a noticeable lack of informa- 18 The Professional Educator Using Vignettes to Assess Teacher Understanding tion on how vignettes were constructed or should the vignette character’s intellectual, cultural, and be constructed. When using vignettes to promote religious background as well as physical attributes discussion and problem solving concerning equity and personal information. They recommended in math and science education, Campbell (n.d.) that vignettes be written so that people from proposed a vignette construction process and different backgrounds could understand them as shared samples of vignettes for different target similarly as possible, that details be scrutinized to groups, (e.g., administrators, teachers, students, avoid introducing unwarranted assumptions, and parents, policy makers, researchers). She listed that vignette characters be as similar as possible to three major steps in creating vignettes: determine the audience. issues and areas of concern, develop situations When authors did provide samples of their that are realistic and relevant, and test the vi- vignettes, a wide range of formats were found that gnettes on groups similar to those who will be included short and long dialogues, formulas, using them (Campbell, n.d.). pictures, and short and long narratives with and In their review of the use of vignettes as a without follow-up questions. Vignettes ranged research tool in the behavioral sciences and health from 25 to 1000 words in length. The