University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln
Public Access Theses and Dissertations from Education and Human Sciences, College of the College of Education and Human Sciences (CEHS)
Fall 10-9-2013
SELF-EFFICACY FOR METALINGUISTIC CONTROL AND ITS RELATIONSHIP TO WRITING QUALITY
Michael S. Dempsey University of Nebraska-Lincoln, [email protected]
Follow this and additional works at: https://digitalcommons.unl.edu/cehsdiss
Part of the Educational Psychology Commons
Dempsey, Michael S., "SELF-EFFICACY FOR METALINGUISTIC CONTROL AND ITS RELATIONSHIP TO WRITING QUALITY" (2013). Public Access Theses and Dissertations from the College of Education and Human Sciences. 191. https://digitalcommons.unl.edu/cehsdiss/191
This Article is brought to you for free and open access by the Education and Human Sciences, College of (CEHS) at DigitalCommons@University of Nebraska - Lincoln. It has been accepted for inclusion in Public Access Theses and Dissertations from the College of Education and Human Sciences by an authorized administrator of DigitalCommons@University of Nebraska - Lincoln.
SELF-EFFICACY FOR METALINGUISTIC CONTROL
AND ITS RELATIONSHIP TO WRITING QUALITY
by
Michael S. Dempsey
A DISSERTATION
Presented to the Faculty of
The Graduate College at the University of Nebraska
In Partial Fulfillment of Requirements
For the Degree of Doctor of Philosophy
Major: Psychological Studies in Education
(Cognition, Learning, and Development)
Under the Supervision of Professor Roger H. Bruning
Lincoln, Nebraska
October, 2013 SELF-EFFICACY FOR METALINGUISTIC CONTROL
AND ITS RELATIONSHIP TO WRITING QUALITY
Michael S. Dempsey, Ph.D.
University of Nebraska, 2013
Advisor: Roger H. Bruning
Currently influential models of writing processes, such as Flower and Hayes (1980) and Hayes (1996) do not attend explicitly to metalinguistics—writers’ ability to monitor and control linguistic skills. Dimensions of metalinguistic ability—metaphonology, metasyntax, metasemantics, metapragmatics, and metatext—arguably are central to the writing process and to writers’ success as they compose. The purpose of this study was to discover if a relationship existed between metalinguistic self-efficacy and (1) ratings on essays written by participants and (2) participants’ self-reported average grade on college papers. Essays were rated using two rubrics, one analytic and the other holistic, which were developed from metalinguistic constructs. The data were analyzed to answer three questions related to metalinguistic self-efficacy and writing quality: (1) exploratory factor analysis was used to determine if five metalinguistic factors would be observed; (2) correlational analysis was used to identify relationships between the metalinguistic self- efficacy measure and the quality of participants’ essays as well as their self-reported college paper grades; and (3) regression analysis was used to determine if the metalinguistic dimensions revealed were equally related to writers’ essay quality and to self-reported college paper grades. The study yielded multiple factors through factor analysis. The correlational analysis revealed an overall relationship between metalinguistic self-efficacy and participant writing quality as well as with self-reported paper grades. Finally, the regression analysis showed that some metalinguistic factors revealed stronger relationships to writing quality than others. Metalinguistic ability, then, was observed to influence writing quality, though not as strongly as anticipated at the start of the study.
ACKNOWLEDGEMENTS
. . . to the composition of [dissertations], nothing is necessary but paper, pens, and ink, with the manual capacity of using them. This, I conceive, their productions shew to be the opinion of the authors themselves; and this must be the opinion of their readers, if indeed there be any such. - Henry Fielding, paraphrased
I hate writing, I love having written. - Dorothy Parker
To Joni, Erin, Kian, and Liam who suffered my absence and neglect far too long as I slouched my way toward my goal. I love you.
To my parents who wanted this for me as much as I did, and for my Dad who won’t be here to see me finish. I love you.
To Roger, Christie, Doug, and Aggie who cared deeply. You know how you helped me; I pray you know how grateful I am.
To my friends, especially Mary and Andy, who gave me generous support when I most needed it. Let’s go have some beers.
To my committee, Roger Bruning, Dave Moshman, Jim Walter, and Kathy
Wilson, whose knowledge, wisdom, and objectivity guided this work to completion.
iv
Table of Contents
Acknowledgements…………………...……………………………………….…….…...iii
CHAPTER 1:
Introduction…………………………….…….…………………………………….....1
Purpose of Study…………..………………………………………………..……11
Research Questions………………………..…………………………..…………12
Definition of Terms………………………...……....………………………….…12
CHAPTER 2:
Review of the Literature.....…………….…………………..…………………....…..14
Flower and Hayes Theoretical Cognitive Writing Model…………………….....16
Metalinguistics and the Flower and Hayes (1980) Model……………………....19
Linguistics and Linguistic Theory...……….……….……..……………...…..….21
Metalinguistics…………………….…………………….…………..………...…24
Metalinguistic Dimensions……………………………………………………....25
How Metalinguistic Dimensions Define Translation in the Writing Process…....31
Example of Metalinguistics During Translation…………………………………35
A Detailed Examination of Writing Self-Efficacy Research……….………...... 36
The Present Study……………………………..…..……………………………..45
Hypotheses……………………….………………………………………………47
CHAPTER 3:
Method……………………………..………………………….……..……………....50
Participants and Setting…………………………………..…....…………………51
Measures and Instruments …………………………………….…………..…..…52 v
Constructing the Metalinguistic Self-Efficacy Scale: The Five Metalinguistic Dimensions and Their Items……………..…….……………….……...... 53
Procedure……………………………………………….…………...……...……57
Research Design…………………………………….……………………………58
Data Analysis…………………………………………………………...……..…59
CHAPTER 4:
Results……………...………………………………………………………………...62
CHAPTER 5:
Discussion…………...…………………………………………………………….…72
Implications and Next Steps…………………………………………………..…79
Limitations to the Current Study……………………………………………..…..82
Conclusion…………………………………………………………………...…..83
References…………………………………….……………………………..…..…...…..84
APPENDIX A. Demographic Survey…………………..…………………..………...... 100
APPENDIX B. Metalinguistic Self-Efficacy Scale..………….………………..…...….101
APPENDIX C. Reading Assignment for Essay.…………………………………...…...103
APPENDIX D. Essay Prompt Used to Generate Writing Sample.…………………..…104
APPENDIX E. Operationalized Rubric for Scoring Participants’ Essays..………….....105
APPENDIX F. Holistic Rubric for Scoring Participants’ Essays…..…………….….....106
APPENDIX G. Recruiting Script..…………..………………………………………....107
APPENDIX H. IRB Document…………………………………………………………108 vi
List of Tables
Table 2.1 The Metalinguistic Self-Efficacy Scale…………………...……………..46
Table 4.1 Means, Standard Deviations, and Skewness for Responses to the Metalinguistic Self-Efficacy Scale, Ratings on Essays, and Self- reported Average Grade on Academic Papers………………………...…65
Table 4.2 Final Factor Loadings for a Two-Factor Model of Metalinguistic Self-Efficacy……………………………………………………………..69
Table 4.3 Correlations for Ideation, Conventions, Ratings, and Self-Reported Average Grade on Academic Papers……………………………………70
vii
List of Figures
Figure 1.1 The Flower and Hayes model (1980)……………………………………..6
Figure 2.1 The revised Hayes model (1996)……………………….……………….18
Figure 4.1 Parallel analysis of factor structure suggesting a two-factor model…….67
1
CHAPTER 1
Introduction
1982 . . . the most promising sign that we are poised for a paradigm shift is that for the first time in the history of teaching writing we have specialists who are doing controlled and directed research on writers’ composing processes. (Hairston, 1982, p. 85)
1992 Maxine Hairston’s 1982 proclamation of a 'paradigm shift’ claimed that the two allied disciplines motivating the new process paradigm were cognitive psychology and linguistics. By the end of the 1980s, one of these forces, linguistics, apparently had vanished. (Faigley, 1992, p, 80)
2012 Despite [an] emerging and converging recognition of the need for models that more meaningfully integrate cognitive and language processes, mainstream writing research still lacks a formal psycholinguistic model of text generation” (Arfé, 2012, p. 573).
The last three decades have produced a body of writing research focused on the cognitive processes activated during writing. Two models are widely recognized as holding particular significance for writing research, the 1980 Flower and Hayes model, and the 1987 Bereiter and Scardamalia model. The models differ in emphasis, the former being a general model of cognitive activity during writing, and the latter being more a developmental model (Alamargot & Chanquoy, 2001). One feature the two models share, however, is a definition of linguistic knowledge. In both models, linguistics is broadly defined as words and syntax, a limited definition, and these are seen as interacting directly with the writing assignment or rhetorical space, though indirectly with the other features of the models (Bereiter & Scardamalia, 1987; Flower & Hayes, 1980; Hayes,
1996). Both models, furthermore, rely heavily on a mechanistic-oriented information 2 processing model that is not linguistic-focused. In the Flower and Hayes model control of writing “inputs” (e.g., the task environment and rhetorical prompt) and cognitive functioning related to writing (e.g., planning and goal setting) describes the complexity of the writing processes (Hacker, Dunlosky, & Graesser, 2009).
The problem is, related to writing, that these models represent a top-down, proposition driven definition of language that obscures the fundamental nature of language production as it likely occurs during writing. What is arguably missing from writing models such as Flower and Hayes’ and Bereiter and Scardamalia’s is language itself—detailed linguistic definitions and processes related to phonology, syntax, semantics, pragmatics, and text (Camps & Milian, 2000; Gombert, 1992; Karmiloff-
Smith, 1981). More importantly to the present study, these models do not attend specifically to writers’ ability to monitor and control these linguistic abilities— metalinguistics. The purpose of this study was to test a metalinguistic self-efficacy scale, which was used to predict ratings on a writing assignment participants completed as well as participants’ self-reported average grade on college papers. The results were envisioned to be helpful in instantiating current writing models with linguistic and monitoring features and to better inform our understanding of the nature of writing, language, and writing process models.
Writing is difficult, especially for novices and underperformers. By all reasonable, objective accounts, young writers’ skills should be of concern (e.g., Bangert-
Drowns, Hurley, & Wilkinson, 2004; National Assessment of Educational Progress, The
Nation’s Report Card: Writing 2012; National Commission on Writing in America’s
Schools and Colleges, 2003; National Writing Project, 2003). Writing underperformance 3 is nothing new, however. In the 1970’s the literacy crisis frightened parents and agitated lawmakers and educators alike. Newsweek writer Merrill Sheils published his alarmist article, “Why Johnny Can’t Write,” stating that the U.S. educational system was “willy- nilly . . . spawning a generation of semiliterates” (p. 58). What should concern us is why, forty years later, it seems little has changed, and we still are in crisis mode. How is it that
Hairston so mistimed her proclamation of a paradigm shift?
Part of the problem, as Faigley (1992) notes, is that linguistics has seemingly been ignored in writing research, at times intentionally. In the early rush to embrace process- oriented research, some writing researchers were quick to toss out “less interesting” aspects of language. Patrick Hartwell (1985), for example, announced that he had abandoned grammar instruction in 1963 after Braddock, Lloyd-Jones, and Schoer concluded that this form of linguistic knowledge, and the instruction used at the time to teach it, was unhelpful, even harmful. Hartwell went much farther, in fact, relying on
Chomsky’s nativist theory of language to defend the abandonment of all research in grammar instruction, “It is time that we, as teachers, formulate theories of language and literacy and let those theories guide our teaching, and it is time that we as researchers, move on to more interesting areas of inquiry” (p. 127). It seems odd that Hartwell and others (cf. Lindemann, 2001; Weaver, 1996) should so readily ignore linguistic ability of this type given that even lower-level abilities such as spelling and punctuation determine clarity and confidence in writing ability, a predictor of actual writing quality (Bruning,
Dempsey, Kauffman, McKim, & Zumbrunn, 2013; Halliday & Hasan, 1976; Halliday &
Matthiessen, 2004; Myhill, 2010; Pajares, 2003, 2007; Shell, Colvin, & Bruning, 1995;
Thompson, 1996). 4
A second problem, as Alamargot and Chanquoy (2012) note, is that we are missing a “unified model of writing processes,” (p. 567) because psycholinguistic and cognitive writing research remain “mutually ignorant,” restricting further progress in the field. Arfé (2012) notes that the “pioneering intuition(s)” of Flower and Hayes (1980) lack the detail of language processes and language mechanisms that a writing process model needs in order to be psycholinguistic. What is missing, in the view of this writer and other linguistic-minded writing researchers (Alves, 2012; Arfé, 2012; Camps &
Milian, 2000; Evans & Green, 2006; Gombert, 1992; Karmiloff-Smith, 1996), is phonology, syntax, semantics, lexics, pragmatics, and text—and the intentional monitoring of these—all linguistic features utilized during the writing process. These constructs are generally referred to as metalinguistics.
Metalinguistics is the field of study that places in the forefront the language-based monitor writers use to control the five linguistic dimensions: phonology (i.e., the sounds of language, which appear in writing as spelling, for example), syntax (i.e., clause structures, which might appear as prepositional phrases or two clauses joined by a conjunction), semantics and lexics (i.e., conceptually related meaning and the words used to represent that meaning), pragmatics (i.e., communicative context and how it shapes and constrains writing, such as the audience or rhetorical context), and text (i.e., cohesion and coherence; the use of pronouns to refer to nouns, for example, and cogency of argument). The present study was meant to help update current writing models by articulating what occurs when pre-linguistic information (i.e., conceptual knowledge) is converted into strings of language drawn from linguistic knowledge. In this study metalinguistic self-efficacy, specifically, was used as a proxy for measuring 5 metalinguistic ability. Two rationales influenced this decision. First metalinguistic self- efficacy is much easier to measure than metalinguistic ability. The latter would require measuring participants’ linguistic attention (e.g. via a think aloud protocol, which raises issues of reactivity and veridicality). Second, self-efficacy and ability are sufficiently correlated that it was assumed that a measure of metalinguistic self-efficacy would yield an acceptable measure of metalinguistic ability (Bandura, 1997; Pajares, 2007).
In current cognitive writing models, these linguistically-related activities are the purview of translation, which Virginia Berninger and H. Lee Swanson (1994) have referred to as being “desperately empty” in the Flower and Hayes (1980) writing model.
Briefly stated, the most influential current writing models are cognitive in nature, and do not explicitly refer to the linguistic skills that are central to translation activities and processes. Translation is the writer’s control over all the constraints attendant to the writing process. I argue that translation processes need to be better specified through a cross-disciplinary approach that draws from both cognitive science and linguistic theory and research. Translation processes, furthermore, should include the notion of intentional monitoring and management of the linguistic tasks inherent to the writing process.
Metalinguistics, specifically, is the intentional monitoring of language objects and processes (Camps & Milian, 2000; Gombert, 1992; Karmiloff-Smith, 1996; Myhill,
2012), and is generally recognized as consisting of five metalinguistic factors: metaphonology, metasyntactics, metasemantics/metalexics, metapragmatics and metatext, which are described in detail later in this chapter. It rests on the five linguistic dimensions described earlier. Metalinguistics is central to this study precisely because it implicates intentional control over linguistic translation processes. Research that analyzes 6 writing ability strictly in terms of general writing processes or outcomes entails primarily declarative and procedural knowledge of writing, which are not sufficient to fully explicate translation. The Flower and Hayes (1980) writing model (see Figure 1.1), identifies a monitor, which is the sentinel poised, for example, to catch and remediate errors or translate into words ideas coming from long-term memory. The perspective of metalinguistics, however, emphasizes similar but more sophisticated linguistic functions: it is intentional monitoring of all language processes during translation and has the potential to shed light on language processes that occur in real time as writers write.
Figure 1.1: The Flower and Hayes model (1980).
If metalinguistics is to be used to better inform the cognitive writing models, empirical evidence is required that the dimensions of linguistics identified above do relate to more general models of writing in some salient way. Flower and Hayes (1980) used think-aloud protocols, and Kellogg (1987, 1993) studied attentional allocation during 7 writing to specify the translation process. Neither of these methods yielded strong linguistic findings (Graham, 2006). The contrasting approach taken in this study was to use a self-efficacy for metalinguistics instrument to measure participants’ confidence for carrying out metalinguistic tasks during writing and compare these to ratings of writing samples and to self-reported grades on college papers. If writers’ self-efficacy for the metalinguistic dimensions turned out to relate to their actual writing quality on an essay, which it should based on moderate-to-strong efficacy-behavior relationships in a variety of domains, including writing (Bandura, 1986, 1997; Bruning et al., 2013; Pajares 2003,
2007), we might be better justified in using metalinguistics to characterize the translation process described in current writing models.
In general, writing self-efficacy research—while not specifically metalinguistic— has focused on writers’ confidence and how it relates generally to writing quality.
Obviously, direct observations of the cognitive activities and linguistically-related decisions being made during writing would be difficult or impossible to obtain. Bandura has stated that self-efficacy predicts engagement and persistence with performance- critical tasks in all domains (Bandura 1986, 1997), supporting the proposition that confidence for metalinguistic abilities can be measured and are likely related to executing these skills during translation. This study attempted to avoid the overgeneralizations and lack of specification found in previous self-efficacy for writing measures (see Bruning et al., 2013 for a discussion of these issues) by creating metalinguistic self-efficacy items that are both (a) specific and relevant to management of the linguistic dimensions of the writing process, and (b) consistent with what generally is called “translation” in current writing models (e.g., Flower & Hayes 1980; Hayes, 1996). 8
The five primary dimensions of metalinguistics are described below along with examples of writing-related activities related to each. These abilities exist on a continuum of cognitive awareness during the writing process and are activated and constrained by several areas of the cognitive writing models, sometimes simultaneously (e.g., in long- term memory as writers access relevant declarative and procedural knowledge, in the task environment where awareness of audience guides the writers in making appropriate metalinguistic choices, and during meaning-making when writers intentionally access semantic knowledge in order to put words on paper). The general assumption that informed the present study was that confidence for intentionally monitoring these abilities, and hence actual ability to carry out metalinguistic tasks while writing, would likely relate to writing quality. In this way, the metalinguistic tasks served as a proxy for predicting writing quality.
First, metaphonology refers to the individual’s intentional monitoring of the sounds of language (see Baddeley, 2007, for a detailed explication of writing, sounds, and the phonological loop), which in writing appear as phonemic and spelling awareness and includes mastery of phonology-related tools needed for writing (e.g., printing words or typing them with a word processor) (e.g., Becker, 2006; Beers & Nagy 2009; Evans &
Green, 2006; Gombert, 1992; Karmiloff-Smith, 1996, McCutchen 2009). Phonological ability, for example, may include choosing words for their sounds (e.g., to create meaning through rhyme). Ability in this area may affect writers’ performance in multiple ways, including the use of figurative writing to create meaning (e.g., rhyming in a poem: Tiger!
Tiger! burning fiercely/In the dark forests versus Tyger! Tyger! burning bright/In the forests of the night, Blake, 1979). 9
Metasyntactics the second major dimension of metalinguistics, refers to the individual’s intentional monitoring of syntax—the rules that control language usage.
This metalinguistic area extends beyond mere punctuation or verb agreement. Writers choose grammatical structures to convey meaning in nuanced and powerful ways (Evans
& Green, 2006; Halliday & Hasan, 1976; Halliday & Matthiessen, 2004; Langacker,
1987, 2008; Thompson, 2004). Syntactic ability varies in choice of appropriate clause structures, for example. Ability in this area may affect writers’ performance in varying clauses for affect, or creating sub-clauses or complex clauses (e.g., He went to the store.
He needed milk versus He went to the store because he needed milk.)
The third major metalinguistic dimension, metasemantic and metalexical ability, refers to the individual’s intentional monitoring of meaning-making via access of the conventional and arbitrary language code, and pairing the two. Following Gombert
(1992), morphology is included in metasemantics. While metasemantics and metalexics are separate dimensions, the former relating to meaning and the latter to words, they are nearly impossible to observe separately during writing (Gombert, 1992). From this point on, therefore, I will follow Gombert and refer to this dimension simply as metasemantics.
In writing, the choice of analogy, simile, exemplification, irony, etc., are examples of metasemantics. These abilities vary between individuals in terms of vocabulary and connotative and denotative knowledge, as well as conceptual knowledge. Ability in this area may affect writers’ performance to choose powerful words or to access appropriate ones for the meaning intended (e.g., The critic overstated his opinion versus The critic stated his opinion in a tone of self-embarrassing jubilance.) 10
Metapragmatics is the fourth major metalinguistic dimension, and it refers to individual’s intentional monitoring of “the ability to represent, organize and regulate the use of speech itself” (Hickmann, 1983, p.21, quoted in Gombert, 1992). Metapragmatical awareness is the choice and manipulation of communicative rules that relate linguistic forms to communicative contexts (Austin, 1975; Bates, 1976; Grice, 1975; Gombert,
1992; Karmiloff-Smith, 1996; Searle, 1979). More precisely, metapragmatics addresses genre and audience driven processes in writing. Ability in this area may affect writers’ performance to develop effective argumentation, that is the ability to use evidence to inform and convince a reader (e.g., The guest overstayed his welcome versus The guest overstayed his welcome in days, cleanliness, and boorishness, the last being his greatest offense. The latter sentence offers specific information that gives detail to help an better understand the writer’s intention).
Metatext, the final metalinguistic dimension, refers to the individual’s intentional monitoring of semantic characteristics, and the textual choices they entail, in consideration of the meaning and relationship of individual sentences to each other and to the text overall. Central to this definition are the ideas of cohesion and coherence
(Halliday & Hasan, 1976; Halliday & Matthiessen, 2004; Thompson, 2004). Ability in this area may affect writers’ ability to make logical connections between sentences and paragraphs and whether the text overall hangs together as a coherent whole (e.g., James went to the ballpark. The hot dog was great versus James went to the ballpark. The hot dog he ate there was great.)
Collectively, from the perspective of this study, these five dimensions of metalinguistics, which have not been studied empirically in writers or in the quality of 11 their writing, are seen as important if not essential to the functioning of writers’ internal editors, the linguistic monitor needed to direct attention to the critical language tasks being carried out during the writing process. That is, intentional monitoring of the five linguistic dimensions is what writers rely on as they isolate and process linguistic forms, recognize and correct errors, and attain writing goals. For expert writers, many writing processes are automatized, but it is clear from how easily automaticity is interrupted that even these expert writers continue to maintain a level of linguistic awareness and knowledge of the choices available for controlling the writing process. Writing quality, I argue, is affected by fluid control and monitoring of metalinguistic tasks during composition. Control and monitoring of this kind, furthermore, requires deep understanding of both declarative and procedural writing knowledge to make solid language choices and identify and instantiate intentions (Camps & Milian, 1999;
Gombert, 1992; Halliday & Matthiessen, 2004; Karmiloff-Smith, 1996; Schleppegrell,
2004; Myhill, 2012; Thompson, 1996). We can garner from the need for control and monitoring that metalinguistic knowledge can be a useful benchmark for distinguishing writing quality. If metalinguistic self-efficacy, which in this study was used as an indirect measure of metalinguistic ability, is predictive of writing quality, we may have an improved basis for characterizing the global construct of “translation” in current writing models.
Purpose of Study
The purpose of this study was to examine the relationship of metalinguistic skills used by college students during writing to actual quality of the writing they produce. In this study, based on consistent relationships noted in 12 prior research between self-efficacy in multiple domains and actual performance in those domains (Bandura, 1986, 1997; Bruning, et al., 2013), self-efficacy for executing metalinguistic skills during writing was used as an indirect measure of actual use of metalinguistic skills during writing. The measure of writing performance was ratings of student performance on a writing assignment, which was judged using rubrics developed from the five metalinguistic dimensions.
Findings from this study are deemed to have aided the understanding of individuals’ control over linguistic processes occurring during translation, something arguably absent in current writing models, as well as providing information about the extent to which metalinguistic activities during writing affect writing quality. The study also was viewed as having the potential to affect writing instruction and learning by identifying more specific metalinguistic activities occurring during translation and how they relate to the quality of writing that is produced.
Research Questions
This study examined three research questions, as follows:
1. Will a measure of writing self-efficacy based on metalinguistic theory
be best described by a single factor or multiple factors?
2. Does metalinguistic self-efficacy relate generally to judgments of
overall writing quality?
3. If there are separate factors for writing-related metalinguistic self-
efficacy, are all equally related to general and specific writing
outcomes? 13
Definition of Terms
1. Linguistics: Language’s form, meaning, and context represent the field of
linguistics. Most commonly five dimensions of language are included in
linguistics: phonology, syntax, semantics, pragmatics, and text. Morphology is
notably absent from the present discussion in line with Gombert (1992) who
viewed this dimension as part of semantics.
2. Metalinguistics in writing: The intentional monitoring of linguistic activity during
the writing process.
3. Self-efficacy: Individuals’ self-beliefs regarding their performance in a particular
domain of behavior. This study focuses on participants’ self-beliefs regarding
performing critical writing tasks, that is, linguistic decision-making carried out in
the context of a writing task.
4. Metalinguistic self-efficacy: a measure of individuals’ confidence for
metalinguistic abilities, which serves in this study as a proxy for metalinguistic
ability.
5. Translation: the process by which writers manage and monitor linguistic tasks and
abilities as they compose more or less successfully.
6. Writing: The elaboration of content, writing it, and modifying it.
7. Writing models: The Flower and Hayes (1980, 1984) and Hayes (1996) models
will be used generally when referring to writing models.
14
CHAPTER 2
Review of the Literature
This study was an attempt to explicitly address the translation process in the
Flower and Hayes (1980) model from a linguistic point of view and was based on the premise that utilizing the concepts of metalinguistics can improve our understanding of writing processes and thus enhance writing instruction (Camps & Milian, 2000; Flower &
Hayes, 1980; Gombert, 1992; Hayes, 1996; Karmiloff-Smith, 1996). The field of metalinguistics provides explicit, operationalized definitions of the linguistic features of writing, definitions that may prove useful in better informing translation processes in the
Flower and Hayes model, specifically (Alamargot & Chanquoy, 2012; Faigley, 1992).
Fayol (2012), for example, has stated that the translation process has been examined only imprecisely, conducted in uncontrolled settings, and studied in isolation. Additionally, this study was based on the assumption that self-efficacy can be used as a way to study metalinguistic activities in relation to writing quality, a research method as yet not employed. The approach utilized in this study was to use a metalinguistic self-efficacy measure that taps metaphonology, metasyntax, metasemantics, metapragmatics, and metatext and relates these to the quality of participants’ written essays that were completed during the study and to their self-reported average college paper grades. If the metalinguistic self-efficacy measure proved to relate to these writing measures, it was assumed possible to better specify the linguistic nature of the Flower and Hayes writing model, specifically translation within the model.
One important goal of writing research is to improve writing instruction and learning to write. In Chapter 1 we saw that efforts to reach this goal have at times fallen 15 short, despite notable achievements and productive advancements in writing research (cf.
Alamargot & Chanquoy, 2001). The Flower and Hayes (1980) cognitive writing process model, along with its revisions (Hayes, 1996), is widely acknowledged as one of the most important contributions to writing research in the last half-century. Despite the multiple contributions the model has made however, it does not go as far as it might in explicating the many language-related processes involved in writing. Although the Flower and Hayes model provides a general framework valuable to understanding writing, some have argued that it is underspecified (Alamargot & Chanquoy, 2001; Berninger & Swanson,
1994; Hartley, 1991). It does not, for example, provide a detailed account of how linguistic activity is involved in the writing processes described in the model, especially translation. Also, for the most part linguistic theory has not been utilized to describe the
Flower and Hayes model, and what has been done needs to be expanded in scope
(Alamargot & Chanquoy, 2012; Alves, 2012; Arfé, 2012). There are important exceptions, however. McCutchen and her colleagues (McCutchen, 1986, 1996;
McCutchen, Green, & Abbott 2008; McCutchen, Logan, & Biangardi-Orpe, 2009), for example, have examined morphology and also looked more broadly at dimensions of discourse knowledge, semantics, and syntax. Michel Fayol (2012) has studied details of the writing process related to the lexicon, spelling, and syntax, though he has not explored higher-level writing activities and abilities such as pragmatics or text cohesion.
We look next at the Flower and Hayes (1980) model and the Hayes (1996) writing model in greater detail, including its strengths and weaknesses and how these related to the current study.
16
Flower and Hayes Theoretical Cognitive Writing Model
The year 1980 was an important one for writing research—in this year Flower and
Hayes introduced a cognitive model of the writing process that remains the standard today (Alamargot & Chanquoy, 2001, Graham, 2006). Their model revealed a rich and tangible domain of investigation for cognitive studies of writing. The great power of the model and its subsequent revision (cf. Hayes, 1996) is that it describes writing as a cognitive, non-linear, recursive process. The Flower and Hayes model has produced copious studies in expert-novice differences (Becker, 2006). These studies in turn have revealed that planning and revision abilities most separate experts from novices, findings that have produced important progress in writing instruction and remediation (Harris &
Graham, 1992; Graham & Harris, 2012; Graham, 2006; Graham & Perin, 2007; Saddler
& Graham, 2007).
The 1980 Flower and Hayes model consists of three components (Flower &
Hayes, 1980; Alamargot & Chanquoy, 2001) long-term memory, task environment, and text generation. Long-term memory contains declarative and procedural knowledge related to the topic, audience, genre, etc. It also includes word and syntactic knowledge.
The task environment is everything that is external to the writer but that influences performance. Text generation includes the general writing process, composed of planning and its subprocesses, all of which are managed by a monitor (see Figure 1 above).
In spite of its many positive impacts, the original Flower and Hayes (1980) model has received a range of criticism that generally points to its underspecification of language features, criticisms that are broadly related to translation processes. Hartley 17
(1991), for example, noted five issues with the model, it (1) lacks specification; (2) is purely descriptive; (3) makes no considerations for the writing medium; (4) describes writers working alone only; and (5) ignores inter-individual differences such as style, emotion, and gender. Kintsch (1987) criticized Flower and Hayes for not considering the writer’s creativity, and Kemper (1987) stated that the model “totally” neglected writers’ goals and motivations, and the influence of different types of text and audiences. Kellogg
(1993, 1994), furthermore, has pointed to the lack of attention to working memory, while
Berninger and Swanson (1994) called attention to Flower and Hayes’ lack of specificity for the translation components for writing—the means by which content is elaborated.
Arguably, many if not most of these criticisms are directly related to metalinguistic features of writing: lack of specification of language, style, creativity and the choice it implies, goals to meet audience needs, genre, and the overarching process of using all these features during translation.
In response to his critics, Hayes (1996) revised his model. Instead of three major components, the new model contains two, the individual and the task environment (see
Figure 2.1). The task environment is much as it was in the previous model, but it now includes a social-interaction component, collaborators, and the composing medium being used for writing. The individual, which replaces the monitor, very broadly addresses the rest of the criticisms directed against the original model. The individual includes motivation and working memory (following Baddeley, 1986). It also moves long-term memory inside the individual; and it reorganizes the reviewing process to include text interpretation (reviewing), reflection (planning), and text production. It is not clear if text production refers to translation or to transcription, or to both. 18
In spite of these changes, in this writer’s view, the revised model continues to suffer from some of the same problems inherent to the original model. As a blueprint, it is naturally broad in scope. Nonetheless, processes in the new model arguably continue to be underspecified. The new model lacks clarity in converging working memory, long- term memory, and the cognitive writing processes. In the revised model, for example, linguistic knowledge resides in long-term memory, but it is unclear how this linguistic knowledge is accessed or used via working memory (perhaps the model could benefit by inclusion of long-term working memory as described by Kellogg, 2001).
Figure 2.1: The revised Hayes model (1996). 19
Linguistic knowledge in long-term memory, furthermore, is only indirectly constrained by the task environment in the new model, whereas the two in reality interact more directly when writers attend to, for example, using the past tense for academic genres
(McCutchen, 2000). Finally, cognitive processes, as it is shown, groups language related activities, processes, abilities, and intentions that have not been given their place—or their weight—in the writing process overall. This is especially true for translation— which is treated in detail below—and appears under cognitive processes in the new model as, presumably, text production. In writing about his new model, Hayes (1996) is unapologetic about not addressing the social aspects of writing: “This is because I am a psychologist and not a sociologist.” Hayes is not a linguist, either, and this may explain why he does not give greater attention to linguistic activities in his model.
Metalinguistics and the Hayes (1996) Model
As the previous section explains, metalinguistic activities are missing from the
Hayes (1996) model, and linguistic activities are as yet underspecified and referred to only broadly. In general, there has been very little metalinguistic research related to translation processes, the space in the model where writers control all of the constraints attendant to writing in order to put words on the page (Alamargot & Chanquoy, 2012;
Alamargot & Fayol, 2009; Alves, 2012; Arfé, 2012; Fayol, Alamargot, & Berninger,
2012a). Some have argued (Alamargot & Chanquoy, 2012; Alamargot & Fayol, 2009;
Arfé, 2012) that the reason there are gaps in the research around translation in the Hayes model is that it has not been well-articulated by researchers in the field. Arfé (2012) acknowledges the importance and benefits of a cognitive approach to writing, but she 20 also recognizes that without a linguistic underpinning, the Hayes model cannot answer a number of critical research questions. Arfé writes:
Nevertheless, a full understanding of writing, including its difficulties and
disorders, must consider in detail the language processes and language
mechanisms underlying the generation, formulation, and production of
written text to communicate thoughts. (p. 573)
As Arfé states, a more explicit integration of language into cognitive writing models will benefit developmental studies and help account for a “broad class of errors in writing due to problems in choosing words; encoding and selecting syntactic and grammatical structures; and programming words, phrases, and sentences, which result in dysfluencies and disruptions” (p.573). While Arfé attends to developmental issues and linguistics in her research, her work is equally applicable to more expert writers and to metalinguistics.
In the same tone, Alves (2012) expressed a need for an interdisciplinary approach to writing research, particularly now as writing research enters the mainstream of psychological study. Alves reflected that the “neglected r is no longer neglected,” in large part because of growing questions about writing and writing processes and deep concerns over the quality of students’ writing (see for example National Commission on Writing in
America’s Schools, 2003, and The Nation’s Report Card: Writing 2012). Alves has pointed out, however, that concepts such as planning, revising, and translating, introduced in the original Flowers and Hayes (1980) model, have received unequal attention in writing research. Translating, according to Alves, has been largely ignored.
Alves (2012) has offered three reasons why translating has not been featured more prominently in research. First, because early studies focused on experts, and translating 21 was considered an automatic process, it was assumed it would be too difficult to study translating in expert writers. Second, Alves points out, thinking about translating has evolved toward a view that translation may be a bidirectional, interactive process between cognitive and linguistic transformation processes (Fayol et al., 2012), a view that is understated in the Hayes (1996) model. Finally, Alamargot and Fayol (2009) have stated that language in writing has been split between a psycholinguistic focus on lexical production and cognitive problem solving, arguing that an integration of the two is now needed.
The purpose of the present study was to attempt to better specify the translation activities that were portrayed in the original writing process models. Translation, as
Berninger and Swanson (1994) have noted, needs further explication: it is not clear nor necessarily accurate to conceptualize, for example, metapragmatic ability (e.g., audience perspective taking, rhetorical choices) as entering translation only after first being formulated through a distinct and detached planning process. According to linguistic and metalinguistic theory, the translation process is much more fluid; it may be better described as an online process that does not require the sometimes unidirectional looping posited in early writing models to achieve writing outcomes. To better understand the difference between the Hayes (1996) model and metalinguistics, we need first to better understand linguistic theory. We turn now to a brief discussion of linguistics and linguistic theory.
Linguistics and Linguistic Theory
Linguistic ability is the heart of writing. Without it writing, and communication in general, could not exist. As obvious as this statement is, the Hayes (1996) model, 22 discussed above, generally underplays and underspecifies the essential role linguistics has in producing writing. Linguistics is important because it describes the language processes essential to the translation process and it is the basis for metalinguistic functioning.
Because of the importance of linguistics to the framework of the present study, a brief review of linguistic thought is useful for understanding how this study evolved.
Linguistics is the study of language form, meaning, and context. Specifically, the domain of linguistics includes phonology, morphology, syntax, semantics, pragmatics, and text. (Note: following Gombert, 1992 and researchers who follow his lead, morphology is viewed here as belonging to semantics, which is viewed as including small meaning units like dis- and –ish.) There are three major linguistic theories: formalist linguistics (e.g., Chomsky’s work), functional linguistics (largely the work of M. A. K.
Halliday and his associates), and cognitive linguistics (largely the work of Ronald
Langacker and Charles Fillmore). Each model has its strengths and weaknesses, the formalist view being perhaps the weakest despite Chomsky’s revolutionizing work in linguistics in the 1950’s and 1960’s, which revealed the cognitive foundation of language. The formalist paradigm, however, has to date largely ignored meaning in language (i.e., semantics, pragmatics, and text) and has failed to discover a widely accepted system that describes the rules that govern the use of language (Searle, 1999).
Chomsky’s formalist theory consists of an abstract set of generalized rules detached from a meaningful context of language usage (Halliday & Matthiessen, 2004; Thompson,
2004).
This study draws its theoretical perspectives, therefore, primarily from functional linguistics and cognitive linguistics, which are very similar in their approaches to 23 linguistic ability. Both regard phonology, syntax, semantics, pragmatics, and text as the legitimate objects of linguistic study. The difference between these two linguistic theories is in emphasis. Functional linguistics emphasizes the intentional choices language users make as they write; it focuses on how pragmatics and text structure lower-level language choices, such as phonology and syntax. Cognitive linguistics also recognizes the intentional choice of the writer, but foregrounds the online (i.e., real time) nature of language production and usage, including lower-level and higher-level abilities.
Linguistics’ importance in this study is its fundamental place in the examination of writing itself.
Generally speaking, linguists traditionally study language for its own sake and are less interested in social or cognitive “epiphenomena.” Functional linguists, on the other hand, have been generally very much concerned with the social aspects of language
(Halliday & Hasan, 1976; Halliday & Matthiessen, 2004; Myhill, 2012; Schleppegrell,
2004; Thompson, 2004). In their view, the social environment, equivalent to Hayes’
(1996) task environment, produces constraints that affect translation in fundamental ways. Writers utilize their metalinguistic abilities to make language choices that control these translation constraints for the purpose of creating shared meaning with their texts.
Choice underscores the significance of intentionality, a critical feature of metalinguistics, helping to tie this linguistic perspective to the Hayes (1996) writing model via the monitor. The importance—and demands—of choice in writing is underlined in the title of
Deborah Myhill’s (2012) chapter on metalinguistics, The ordeal of deliberate choice.
Cognitive linguists also work to describe and explain the systematicity, structure, and functions of language; unlike formalist linguists, however, cognitive linguists assume 24 that language reflects patterns of thought, and they therefore study patterns of conceptualization. This assumption leads cognitive linguists to view language as a window into human cognitive functioning, its “nature, structure and organisation of thoughts and ideas” (Evans & Green, 2006, p. 5). Cognitive linguistics differs from other linguistic approaches in that language is assumed to mirror fundamental properties and design features of the human mind (Evans & Green, 2006; Croft & Cruse, 2009;
Langacker, 2008). A cognitive approach to linguistics is consonant with the cognitive nature of the Hayes (1996) model, and research in cognitive linguistics is well-suited for specifying the processes and tasks that underlie translation processes.
This discussion of linguistic theory is pertinent to the current study for two reasons. Linguistic ability is the declarative and procedural knowledge necessary for writing, and linguists differ in how they study language ability and processes. The theoretical linguistic viewpoint used in this study utilizes both functional and cognitive linguistics, despite their differences in emphasis: both are very similar in their approach to linguistics and language; both emphasize the cognitive and intentional nature of language production; and both recognize translation as an online process, one that occurs in real-time without artificial boundaries between individual linguistic processes. We turn now to a discussion of metalinguistics, the intentional control of linguistic declarative and procedural knowledge.
Metalinguistics
This section focuses on the role of intentional linguistic monitoring that affords writers the control needed to balance the numerous linguistic functions and processes that arise during the translation process. As previously stated, metalinguistics is the conscious 25 management of linguistic abilities. There are five metalinguistic factors— metaphonology, metasyntax, metasemantics, metapragmatic, and metatextual—that correspond to the five linguistic features mentioned above in the Linguistics section.
Following the lead of most modern metalinguists I borrow the definitions and construct structure for metalinguistics laid out by Jean Émile Gombert (e.g., Camps, Guasch,
Milian, & Ribas, 2000; Camps & Milian, 2000; Gombert, 1992; Myhill, 2012).
In considering these factors, it is important to be mindful of just what stands for metalinguistic activity. Again, I utilize Gombert’s definition: “From this point of view, metalinguistic activity is characterized by an intentional monitoring the subject applies to the processes of attention and selection [that] are at work in language processing” (1992, p.3) (cf. Camps & Milian 2000; Jakobsen, 1960; Roth, Speece, Cooper, & la Paz, 1996).
It is worth noting that Gombert does accept automaticity of writing skills, though these may be interrupted by situational factors, such as awareness of a typing error. In these instances, intentional monitoring is reactivated. Given the definition above, metalinguistic intentional monitoring will always occur primarily as an explicit process within the cognitive domain during the writing process, which is very much in harmony with the Hayes (1996) writing model and descriptive of the translation process.
We turn now to detailed definitions of the five metalinguistic dimensions, including critical tasks research indicates are appropriate for measuring each. Each of these dimensions can be viewed as descriptive dimensions of the translation process.
Metalinguistic Dimensions
The first of the five metalinguistic dimensions is metaphonological ability, the skills needed to identify the phonological components in linguistic units and intentionally 26 manipulate them. These include a range of meaning-making units of language in writing, a definition that is shared by numerous research fields, including metalinguistics, cognitive linguistics, developmental research, and cognitive psychology (e.g., Beers &
Nagy 2009; Evans & Green, 2006; Gombert, 1992; Karmiloff-Smith, 1996, McCutchen
2009). Three critical tasks typically define this dimension. The first is phonemic awareness, the ability to recognize and use phonemes during the writing process, a skill necessary for spelling accuracy (Gombert, 1992; Kamiloff-Smith, 1996; McCutchen,
1996, 2009; Myhill, 2012). The second ability is spelling itself, a specifically writing- related task (Berninger, Fayol, & Alamargot, 2012, Kellogg, 1994; McCutchen, 1986,
2000; Myhill, 2012) (For alternative views, see Elbow, 1994; Schleppegrell, 2004). The third ability, the writing medium (i.e., paper and pencil, computer word processor), is overlooked in the Flower and Hayes (1980) model (Hartley, 1991), but appears in Hayes’ revised model (1996), though again without comment or specification despite the wide body of research suggesting that individual differences in handwriting, for example, predict writing achievement (Becker, 2006; Graham, Berninger, Abbott, Abbott, &
Whittaker, 1997). The relationship of writing medium to the cognitive writing process is now widely studied as it relates to computer word processors (cf. Becker, 2006; Hartley,
1991; Kellogg, 1996).
Metasyntactic ability, the second metalinguistic dimension, is the ability needed to consciously reason about syntax as well as deliberately and accurately control grammar usage, the two critical tasks typically used to define this dimension. In writing, decisions about syntax and grammar are especially critical because these choices affect not only the form of writing but the meaning as well (Christie, 2010; Gombert, 1992; 27
Halliday & Hasan, 1976; Halliday & Matthiessen, 2004; Karmiloff-Smith, 1981;
Schleppegrell, 2004; Saddler & Graham, 2007). As functional linguistics suggests, syntax is meaning, and is most clearly observable in writing than in speech where context plays a greater meaning-making role. Metasyntactic ability relies on explicit syntactic knowledge and overlaps somewhat with higher-level abilities. Functional linguistics recognizes a symbiotic relationship between grammar and syntax in much the same way that Gombert treats metasemantic ability as a singularity (Christie, 2010; Halliday &
Matthiessen, 2004; Schleppegrell, 2004). This metasyntactic ability, furthermore, is often constrained by higher-order metalinguistic abilities, according to functional linguists (we begin to see more clearly here the way in which higher-order metalinguistic abilities interact with lower-order ones; Halliday & Matthiessen, 2004; Thompson, 2004).
The third metalinguistic dimension, metasemantic ability includes the knowledge that words are words and the skill to select words intentionally during the writing process. More so than in speech, metasemantics in writing has the potential to move readers precisely because the writer has more time to choose the best word or semantic strings. Three critical tasks typically define this dimension. Word choice is the ability to intentionally enter the lexicon to select vocabulary appropriate for the given writing context (Bereiter & Scardamalia, 1987; Fayol, Alamargot, & Berninger, 2012b; Flower and Hayes, 1980; Geeraerts, 2010; Gombert, 1992; Hayes, 1996; Karmiloff-Smith, 1996;
Lakoff & Johnson, 1980; Langacker, 2008; McCarthy, Briner, Myers, Grasser, &
McNamara, 2008; McCutchen, 2012; McNamara & Grasser, 2011; Myhill, 2012). It is also the ability to reflect on knowledge of the concepts words point to—both connotatively and denotatively (Croft & Cruse, 2004; Evans & Green, 2006, Langacker, 28
2008). Choosing how and when to use words during the writing process—in singularity or in strings—is at the heart of semantics. Individuals’ semantic ability refers to their skill in creating a shared meaning space with their audience (Fauconnier, 1994) and is often treated as propositions. Propositions represent the smallest units of meaning that can stand as a single assertion, for example, The Boston Red Sox won the 2004 World Series
(J. R. Anderson, 1996, 2000; Kintsch & van Dijk, T. A.; Rumelhart & McClelland,
1985). Word choice and propositions generally interact during writing, working together to create meaning. After all, our word choices often affect the meaning intended, and this ability in writing is often seen as a mark of competence.
The fourth metalinguistic dimension is metapragmatic ability. In writing, this is the ability to reflect on language use in context to achieve meaning (e.g., genre, reflecting on one's audience, reflecting on rhetorical options). Pragmatics in writing is the context that helps structure the meaning-making process. It differs from semantic and syntactic knowledge by considering how perspective-taking and intended meaning affect writing
(Austin, 1970; Grice, 1975; Fauconnier, 1994; Searle, 1969). Intentional monitoring in this domain requires that the writer understand a great deal about the intended reader:
What is the reader’s attitude toward my topic and me? What information does the reader currently have regarding my topic? And most importantly, how do I tie my syntactic and semantic knowledge to best present my topic and ideas (Bereiter & Scardamalia, 1987;
Crowhurst, 1990; Flower & Hayes, 1980; Halliday & Matthiessen, 2004; Gombert, 1992;
Kamiloff-Smith. 1996; McCutchen, 2000, 2008; Nippold, Ward-Lonergan, & Fanning,
2005; Rubin & Piche, 1979; Thompson, 2004)? The last question most clearly identifies the intentional nature of metapragmatic activity in writing and points to its framing 29 nature: metapragmatics requires writers to consider the genre and rhetorical form they will use to structure metaphonology, metasyntax, and metasemantics in a way that best guides the reader to a shared mental space (Evans & Green, 2006; Fauconnier, 1994;
Lakoff & Johnson, 1980). Three critical tasks typically define metapragmatic ability. The first, audience perspective taking, involves the writer consciously identifying the intended audience, how this audience thinks, feels, and what this audience knows about the writer’s topic (Chenoweth & Hayes, 2001; Bruning & Horn, 2000; Flower & Hayes,
1980; Gombert, 1992; McCutchen, 2000; McCutchen & Perfetti, 1983; Myhill, 2012;
Schleppegrell, 2004). The second, recognizing the clarity of one’s argument, occurs when the writer initiates the internal editor in order to judge the appropriateness of the argument in process. It is a difficult task because it generally occurs in isolation: writers must rely strictly on their knowledge of their audience to know if they have expressed themselves accurately, fully, and convincingly (Rijlaarsdam & Couzijn, 2000; Nippold et al., 2005). The last ability, recognizing the quality of one’s rhetorical goals, is the activity where the writer selects the most appropriate genre and argument format to present a topic. It relies on audience knowledge and on some idea of the semantic content to be used (Chenoweth & Hayes, 2001; Bruning & Horn, 2000; Flower & Hayes, 1980;
Gombert, 1992; McCutchen, 2000; McCutchen & Perfetti, 1983; Myhill, 2012;
Schleppegrell, 2004).
Metatextual ability is the final metalinguistic dimension and includes syntactic, semantic, and pragmatic abilities that ensure that the meaning between text structures hangs together both in part and as a convincing whole. In speech, context requires only loose control over metatext because context fills in where words do not. This is not true 30 in writing. Metatextual ability in writing refers primarily to coherence and cohesion.
According to Thompson (1996), cohesion refers to linguistic items that are repetitive textual devices; they are words that allow writers to vary their writing for interest or other intentions, and they must point to their referents clearly in order not to confuse the reader.
Coherence, on the other hand, is a mental phenomenon much in the sense that Fauconnier
(1994) intends with his concept of shared mental spaces. It refers to the clarity of shared meaning represented in the text in part or in whole. Metatextual ability overlaps and incorporates declarative and procedural knowledge from lower order metalinguistic abilities (Gombert, 1992; Karmiloff-Smith, 1996; Myhill, 2012; Rijaarsdam & Couzijn,
2000). Two critical tasks typically define this dimension, cohesion and coherence. I offer two examples for the first construct, cohesion, though this ability is explored more extensively in the fields of functional and cognitive linguistics (Croft & Cruse, 2009;
Evans & Green, 2006; Halliday & Hasan, 1976, Halliday & Matthiessen, 2004;
Schleppegrell, 2004; Thompson, 1996). The first example is substitution in writing, in this case pronouns for nouns: “Bob willed the schooner toward the port. He knew two masts were gone altogether.” Second is ellipsis, the elimination of text not necessary for meaning given the context already provided: “‘What’s the state of the rudder,’ Bob called. ‘Gone,’ was the Boatswains’ reply.” The second construct, coherence, refers to cogent writing that engages the reader (Halliday & Hasan, 1976, Halliday & Matthiessen,
2004; McCutchen & Perfetti, 1983; Schleppegrell, 2004; Thompson, 1996). While simple in definition, and easy to recognize, it is perhaps the most difficult skill for writers to master. It is a higher-level metalinguistic ability that requires the writer to tap, sometimes simultaneously, all of the metalinguistic skills listed above. 31
How Metalinguistic Dimensions Define Translation in the Writing Process
The definitions and examples above suggest that metalinguistic dimensions play a role during translation. I now assert that all five dimensions are, in fact, necessary for a complete explication of translation in the writing process. Metaphonological ability, for example, is critical to academic writing quality. Research has shown, for example, that in young children spelling quality is slow to develop in relation to their writing, affecting the quality of what they write (e.g., Berninger, 1996; Berninger, et al., 2012; Kellogg,
1994; McCutchen, 2000; Myhill, 2012). Spelling can affect young writers’ writing quality because children must map the language system to an arbitrary writing code during the translation process. Even among older writers, however, there is evidence that spelling predicts writing quality. Spelling related items on self-efficacy scales (Bruning et al., 2013; Pajares & Valiante, 1997, 1999, 2001, 2006; Shell, Colvin & Bruning, 1995) have shown that spelling confidence is a predictor of writing quality, with poor spellers performing worse on writing outcomes than more competent spellers.
The dimension of metasyntax is also central to the quality of writing outcomes. In children, sentence structures, for example, are often simplistic and repetitive, though these improve developmentally with age (Camps et al., 2000; Gombert, 1992; Karmiloff-
Smith, 1996; Myhill, 2012; Saddler & Graham, 2007). In adults, self-efficacy research has also utilized task items that are syntactic in form to show a relationship between confidence and quality of writing outcomes (Bruning et al., 2013; Pajares & Valiante,
1997, 1999, 2001, 2006; Shell et al., 1995; Shell et al., 1989). Shell et al (1995), for example, asked participants to rate their confidence to “write compound and complex sentences with proper punctuation and grammatical structure.” They found that syntactic 32 ability was highly related to writing success. Again, whether young or adult, writers’ syntactic ability affects their writing outcomes.
Metasemantic ability has also been shown to affect writing quality. This ability can distinguish a competent writer from an excellent writer: being able to select the mot juste and apposite semantic strings certainly supports meaning-making and the reader’s understanding and attitude toward the writer. Domain knowledge, for example, has been shown to relate to use of more sophisticated word meanings in children (Bereiter &
Scardamailia, 1987; McCutchen, 1996, 1997; Myhill 2012). In adult writers, McCarthy et al. (2008) found individual differences in participants’ metasemantic ability. They showed that participants who could recognize genre within the first three words of a sentence had higher language abilities, broadly measured. Though not explicitly stated, the authors were asking participants to tap the moment word choice and semantics triggered metapragmatic knowledge. The researchers’ findings suggest that metasemantics does show language-related individual differences. Reading, however, is not writing, and caution must be taken in applying these results to the current study. The
McCarthy et al. study does suggest, however, that there are habitual word meanings that are genre specific, that is word choice can support metapragmatics and writing quality overall.
Metapragmatics is one of the most difficult dimensions to master (Bereiter &
Scardamalia, 1987). Metapragmatics “strongly governs and controls the actual writing process and product. Good writers—and good writing students—invest in getting a clear picture of the task conditions, and apply it in their writing” (Rijaarsdam & Couzjin, 2000, p. 171). Evans and Green (2006), moreover, use Fauconnier’s theory of mental spaces to 33 describe the nature of meaning construction in language as a function of pragmatic ability:
. . . linguistic expressions have meaning potential. Rather than
‘encoding’ meaning, linguistic expressions represent partial building
instructions, according to which mental spaces are constructed. (p. 371,
bold in original)
In speech, the audience provides cues that act as easy aids to communication, such as facial features or question asking. Mastery of metapragmatic ability in writing, however, relies on writers’ prowess in understanding and communicating with an audience that is not present (Bereiter & Scardamalia, 1987; Flower & Hayes, 1980; McCutchen, 2000,
2008), and this prowess is a mixed bag in college-age students (Grasser, Hopkinson,
Lewis, & Bruflodt, 1984; Fallahi, Wood, & Austad, 2006). According to Graesser et al.
(2010), there are two hypothetical constraints that make metapragmatic ability difficult for college writers, producing greater variance among them. First is the capacity limitation hypothesis in which writers throw out or limit pragmatic constraints due to cognitive pressure in working memory. Second, the authors suggest the bankrupt idea generation hypothesis, according to which “ . . . it is difficult for writers to generate ideas that are informative, interesting, sophisticated, and relevant to a particular pragmatic context” (p. 361). Regardless whether these hypotheses fully explain metapragmatic differences, they do suggest ways in which task ability in this domain might affect writing quality.
Metatextual ability represents writers’ control over the level of interest they can generate in their writing as well as the control over connecting sentences and paragraphs 34 into a cohesive, coherent whole. In children and adults, cohesion affects the quality of writing outcomes. Children who have not yet mastered conjunctions, for example, do not produce engaging text (Beers & Nagy, 2009; Gombert, 1992; Karmiloff-Smith, 1996;
Myhill, 2012; Saddler & Graham, 2007, Swierzbin, 2010). Adults are more adept with substitution, ellipsis, and conjunction—simpler forms of coherence—but they still show difficulty with, for example, demonstratives (e.g., these, those, Gombert, 1992;
Karmiloff-Smith, 1996; Myhill, 2012; Swierzbin, 2010; Thompson, 2004). Coherence, however, can overwhelm young writers (Gombert, 1992; Karmiloff-Smith, 1996; Myhill,
2012; Swierzbin, 2010) and adult writers alike (Fallahi et al., 2006; Grasser et al., 2010).
In short, crafting extended academic prose that is well organized, cogently and engagingly written is extraordinarily difficult.
Writers differ in their linguistic ability, and the same is true of their metalinguistic ability. Metalinguistic research suggests that there are individual differences in all areas of metalinguistic ability (Badia, 2000; Camps et al., 2000; Camps & Milian, 2000,
Gombert, 1992, Karmiloff-Smith, 1981, 1996; Rijlaarsdam & Couzijn, 2000).
Differences in writing ability related to the translation process have in the past been described using think-aloud and attentional protocols (Gombert, 1992; Flower & Hayes,
1980; Kellogg, 1987, 1993). Think aloud protocols, however, can be intrusive and time consuming and are subject to issues of accuracy and reactivity that can occur when
“thinking about thinking.” This study utilized a different approach of measuring metalinguistic ability by administering a metalinguistic self-efficacy instrument to participants and relating their answers to the quality of the essays they produced during the study. This approach is supported by the findings of Bandura (1986, 1997) and many 35 others (e.g., Pajares, 2003; Shell et al., 1995; Shell et al., 1989; Zimmerman & Bandura,
1994) who have shown that self-beliefs related to confidence in a particular domain are consistently related to successful performance in that domain. Alves (2012) offers his support for just such an approach in his call for interdisciplinary research in the area of writing motivation, which he feels has been understudied. There is, he writes, a need to collaborate with researchers in other disciplines that study “emotions, feelings, moods, attitudes, beliefs, motivations, and evaluations to situate these affect variables within the writing process” (p. 594). We turn now to an example of how metalinguistic abilities might play out during the translation process.
Example of Metalinguistic Activity during Translation
An example may help explicate the metalinguistic nature of translation as it appears in the Hayes (1996) model. Imagine a young writer faced with the task of composing an academic paper. With respect to the model, this writer may begin anywhere within it, but most likely would first consider the topic and access declarative knowledge related to it. At this point, the student will likely generate semantic strings, some short and some longer. At the same time, the student could be expected to intentionally access metaphonological knowledge, for example knowledge that academic writing often uses specific verb tenses, which could prompt the writer to think intentionally about morphemes, which are within the realm of metasemantics in this view. The writer most likely would continue to direct attention to the task environment and continue to intentionally enter the lexicon and monitor phonological strings needed to meet tense requirements. The writer could also at this time begin to monitor audience perspective and the genre selected to check how the words chosen support these linguistic 36 dimensions. Audience and genre monitoring could create more intentional monitoring of semantics and phonology. When the writer becomes aware of a disjunction between goal and text produced, however, (e.g., incorrect morphemes) metaphonological activity may begin again (i.e., the intentional access of phonological knowledge in long-term memory will start again). From this example we can see that none of these dimensions is accessed in total isolation; they can be, however, parsed to better understand what is happening during translation.
As the example above suggests the translation process may not be as reliant on planning or goal setting as Hayes implies in his 1996 model. The Hayes model remains a top-down approach to the writing process, and is subject to criticism (Hartley, 1991), just as the artificial intelligence models upon which it is based have been similarly criticized
(e.g., ACT-R). The reality is that such a highly plan-driven approach is true only in the most artificial and highly structured writing situations (Berninger & Swanson, 1994;
Halliday & Matthiessen, 2004; Thompson, 2004). Thus, the Hayes model does not fully represent the interactive nature of metalinguistic ability and its relationship to the translation process.
A Detailed Examination of Writing Self-Efficacy Research
If one assumes that metalinguistics can help better specify translation processes in the Hayes (1996) model, it is necessary to look for appropriate ways to measure writing quality that are closely tied to metalinguistic choices and behaviors. Self-efficacy was chosen as the tool to tap individuals’ confidence to perform successfully in the domain of metalinguistics, and to relate metalinguistic performance to writing quality.
Metalinguistic self-efficacy was measured using a scale, which in turn was used as a 37 proxy for predicting success on the writing assignment used in this study and self- reported average college paper grades. While research on writing self-efficacy has shown consistent relationships between writing self-efficacy and writing quality generally
(Pajares 2003, 2007; Shell et al., 1995; Shell et al. 1989), it has been tied to metalinguistics only tangentially.
The research that has been done for metalinguistic self-efficacy has been sparse, indirect, and frequently framed mostly within the context of English as a Second
Language and English as a Foreign Language as confidence for using instructional strategies. Rodriguez (2006), for example, indirectly connected self-efficacy to growth in metalinguistic awareness for English in her action-research dissertation, as did Matthews
(2010) in his research on ESL tutoring. Both researchers were concerned primarily with measuring the robustness of instructional methods, not with the self-beliefs related to metalinguistic awareness and whether these can predict writing quality. In the area of metalinguistic development, furthermore, some research designs have examined motivation generally, though not self-efficacy specifically (cf., Hall, Smith, &
Wicaksono, 2011). Karmiloff-Smith (1981, 1996) and Gombert (1992) have used feedback as an indirect measure of metalinguistic development, a key factor in developing self-efficacy. Their four-tier, Piagetian developmental process does not, however, focus specifically on individual self-efficacy; instead these researchers were concerned with the acquisition of metalinguistic skills as opposed to competent use of them. Finally, Rijlaarsdam and Couzijn (2000) encouraged prompting students to utilize self-reflection and self-feedback during writing with respect to metalinguistic skills such as syntax construction, but went only so far as to state that “learning processes rely on 38 this input” (p. 176). It is significant that, as indirect as it is, only Rijlaarsdam and Couzijn
(2000) reference the dimensions of awareness, metalinguistics, and writing quality simultaneously. The present study, in contrast, focused directly on metalinguistic processes occurring in writing, and used a newly constructed measure of metalinguistic self-efficacy designed to represent each of the major dimensions of metalinguistics. We turn now to an in-depth review of writing self-efficacy research to better understand how the present study builds upon and extends previous research methods.
Few self-beliefs have proven more important to individual performance than self- efficacy beliefs, the confidence that one can perform successfully in a given domain
(Bandura, 1997, 2006; Bruning et al., 2013). Individuals with high self-efficacy engage domain activities more quickly and persist longer in the face of obstacles. Indeed individuals with higher self-efficacy tend to set higher goals, use more effective learning strategies, and have lower anxiety while engaging in domain activities (Bong, 2006).
These positive outcomes are particularly important when individuals find themselves faced with critical, ill-defined tasks that can diminish motivation (Bruning & Horn,
2000). Writing, specifically translation during writing, is one such ill-defined domain, and the current study was deemed useful for tapping metalinguistic self-efficacy during translation.
Writing self-efficacy research has explored a wide range of writing contexts and writing-related variables (Bruning et al.; 2013; McCarthy, Meier, & Rinderer, 1985;
Pajares, 2003, 2007; Pajares & Johnson, 1996; Pajares, Miller, & Johnson, 1999; Pajares
& Valiante, 1997, 2006; Shell et al., 1995; Shell, Murphy, & Bruning, 1989; Zimmerman
& Bandura, 1994), the results of which have been used to develop models and new 39 instructional strategies, and to describe features of diverse populations of writers. Despite these successes, writing self-efficacy research has been slow to create critical performance task items that are related to writing prima facie. Part of the problem is that research in this field often rests on weak theories of writing, or no theory at all, nor has it connected to writing models that might offer the possibility of doing so (see Bruning et al., 2013, for examples).
One of the earliest studies of writing self-efficacy was conducted by McCarthy and her associates (1985). They studied the relationship between college students’ writing efficacy and their writing performance. McCarthy et al. picked 19 skills they felt represented abilities students utilize in academic writing. Participants rated their self- efficacy for matching these same skills. The authors described their items as related mostly to writing mechanics, such as spelling errors and run-on sentences, not higher order linguistic skills or metalinguistic ability. McCarthy et al. found that 10%-15% of the variance of students’ actual writing scores was explained by their self-efficacy scale.
Shell et al. (1989), in a follow-up study on McCarthy et al.’s 1985 work, surveyed college students’ perceptions of the relationships between writing self-efficacy, outcome expectancy beliefs, and writing achievement. They gathered writing samples that were evaluated analytically by two raters blind to the students’ survey responses. Writing efficacy was measured with two subscales: a task subscale and a component skills subscale. The first consisted of items sampling efficacy to different writing purposes or tasks, such as writing an essay, magazine article, or short story (i.e., genre related outcomes). The second subscale, much like the McCarthy et al. scale, asked students to rate their confidence for mechanics-related skills, such as spelling, using parts of speech, 40 and punctuation. Shell et al. (1989) found that writing task self-efficacy did not predict writing performance, but writing skills efficacy did. Again, higher-order metalinguistic items were not included on the scale.
In a subsequent study, Shell et al. (1995) looked at differences for achievement level and writing self-efficacy at grades 4, 7, and 10—what amounted to one of the first developmental studies in this area of research. Brief essays were collected from students, and students were given self-efficacy scales similar to those used in the 1989 study, but adapted for younger writers. Writing skills self-efficacy was shown to predict writing performance at all grade levels but did not increase significantly with grade level. Writing task self-efficacy, as in Shell et al. (1989), did not prove significant.
Bruning et al. (2013) have remarked that “writing self-efficacy gains are more related to students’ abilities to successfully perform various writing tasks than to changes in specific writing skills” (p. 26). Alternatively, we may conclude that the Shell et al.
(1989, 1995) skills subscale was not sensitive enough to find developmental differences, and that it did not account for younger students’ propensity to overrate their own skill level. Young writers, especially, should lack confidence for mechanical skills precisely because they have had little experience with or feedback on writing skills. The tenth graders in the study, furthermore, should on average show more confidence in this domain because they have had ten years experience with academic language and nearly as much with writing. It is possible that Shell and associates might have found more robust developmental findings had they primed students (i.e. shown them ahead of time what poor and good linguistic ability looks like, Bandura, 2006), and if they had asked 41 them to rate their confidence to control metalinguistic abilities during translation as opposed to task abilities (e.g., I can write a magazine article).
Pajares and his associates have produced one of the most comprehensive writing efficacy programs to date (e.g., Pajares, 2003, 2007; Pajares, Britner, & Valiante, 2000;
Pajares et al., 1999; Pajares & Valiante, 1997, 1999, 2001, 2006). In an early study,
Pajares & Johnson (1994) found that writing skills predicted students’ performance on an essay, but not writing task self-efficacy, replicating Shell et al. (1989. 1995). Pajares and
Valiante conducted several additional studies that found (a) writing self-efficacy related to gender-related differences, (b) an inverse relationship to writing apprehension, and (c) its independent and mediating effects on writing achievement (Pajares & Valiante, 1997,
1999, 2001, 2006).
Throughout their research, Pajares and Valiante have utilized the Writing Self-
Efficacy Scale (WSES), which they developed (Pajares, 2007; Pajares & Valiante, 1999).
Items on the WSES scale, according to Pajares, were broadly related to items used by
Shell et al. (1989, 1995), which they modified to make grade-level appropriate.
According to Pajares (2007), the WSES asks students to “provide judgments of their confidence in their ability to successfully perform grammar, usage, composition, and mechanical writing skills, such as correctly punctuate a one page passage or organize sentences into a paragraph to clearly express a theme” (p. 240). These skills represent an advance over Shell et al. (1989, 1995): items from the WSES tap syntactic and textual domains in addition to simple mechanics. Still, the WSES is not fully metalinguistic in nature. It does not include items related to metasemantics or metapragmatics. The item on paragraphs may be considered, however, related to metatextuality. 42
Pajares and associates most often treated writing self-efficacy as a unidimensional construct. Importantly, however, Pajares (2007) tested whether the WSES might reflect separable dimensions of writing self-efficacy. He conducted exploratory factor analyses of the writing performance of students Grades 4 through 11. Two factors emerged: a basic, low-level writing skills dimension (e.g., spelling, punctuation, and verb tense), and a more complex level of writing skills dimension (e.g., structuring paragraphs and essays, using topic sentences). Pajares’ findings here are important to the current study because they suggest that writing self-efficacy can be parsed into separable, more meaningful factors, which has the potential for more closely identifying writing skills that are critical to students’ writing self-efficacy and can be related to their writing quality.
Still, Pajares and his colleagues and Shell and his associates did not tie their measures to known writing models (e.g., e.g., Bereiter & Scardamalia, 1987; Flower &
Hayes, 1984; Hayes, 1996, 2006) or to potentially writing-relevant psychological and language-related processes (e.g., Flower & Hayes, 1984; Myhill, 2008: Schleppegrell,
2007). Their work lacks a degree of face validity because their constructs are atheoretical in relation to language and the control of language during the writing process.
Zimmerman and associates have been more successful in tying their work to writing models and the writing constraints, skills, and tasks related to translation. For example, in an early study, Zimmerman and Bandura (1994) predicted that college students’ self-efficacy for regulating writing activities along with self-efficacy for academic achievement and grade-related goals would predict writing attainment beyond verbal aptitude. The researchers recruited English composition students to complete their
Writing Self-Regulatory Efficacy Scale. The scale contained 25 items that tapped 43 strategic aspects of writing such as planning and revising, responding to creative elements in writing, and self-managing writing activities and time on task. Principal components analysis revealed a single factor for the Writing Self-Regulatory Efficacy
Scale. Amongst their findings, Zimmerman and Bandura found that students in advanced composition had higher self-efficacy for managing writing tasks, and self-efficacy predicted self-evaluation and confidence for receiving higher grades. Significantly,
Zimmerman and Bandura concluded from this study that their scale had the potential for diagnostic assessment. The ability to tap writing self-efficacy as an assessment tool for placement and remediation is clearly advantageous to school writing programs.
Bruning and associates (2013) recently extended previous findings in writing self- efficacy by developing a measure based on specific cognitive (Bereiter & Scardamalia,
1987; Flower & Hayes, 1980; Hayes 1996) linguistic (Cruse, 2004; Evans & Green,
2006; Langacker, 2008) and self-regulatory (Zimmerman & Bandura, 1994; Zimmerman
& Kitsantas, 2002) writing tasks and activities. From this theoretical base, the researchers hypothesized a three-factor structure for writing self-efficacy: Ideation, Conventions, and
Self-Regulation. Ideation broadly corresponds to semantic tasks in writing and includes items such as I can think of a lot of original ideas, and I can put my ideas into writing.
Conventions is similar to the work of Pajares and corresponds broadly to syntax. It includes items about punctuation, for example, but also one item on text structure (I can begin my paragraphs in the right spots). Self-Regulation in Bruning et al. (2013) is very much tied to the work of Zimmerman and his colleagues and corresponds to the idea of cognitive monitoring. From this base, Bruning et al. (2013) developed the Self-Efficacy for Writing Scale (SEWS), which comprised 16 items. 44
To test their hypothesis, Bruning et al. (2013) conducted two studies, one with middle school students and one with high school students. Six hundred ninety-seven middle school students were included in the first study, all enrolled in 8th Grade. The researchers first fit the data to a single-factor model, which was unacceptable. Turning to the three-factor model, they found it was an acceptable model for the data, significantly better than the one-factor model, and it encompassed all 16 items. In the second study, which included five hundred sixty-three high school students, confirmatory factor analysis was used to test the generalizability of the three-factor model revealed in the first study. As in the middle school study, the high school data supported a multifactor conceptualization of writing self-efficacy. In these studies, Bruning et al. (2013) extended thinking about writing self-efficacy significantly. The clear and strong results they found are most likely due to the unique and original theoretical conceptualization that underlies the SEWS and the two studies.
In summary, writing self-efficacy research, extending back to the 1980’s, has revealed consistent findings showing a predictive relationship between writing self- efficacy and writing skills (e.g., punctuation and grammar). Pajares and associates’ research indicated that rewriting items to include higher-order writing tasks yields a more sophisticated factor structure for writing self-efficacy, and Zimmerman and Bandura
(1994) discovered a close link between monitoring self-efficacy and writing efficacy.
Most of the measures used in these previous studies, however, tapped broad and indistinct dimensions of writing self-efficacy. Bruning et al. (2013) found that “no models or measures have been developed that provide independent information on writers’ self-efficacy for meeting writing’s cognitive and linguistic, as well as self- 45 regulatory demands” (p. 27). Bruning et al., however, have broken new ground in writing self-efficacy research by looking specifically at linguistic and self-regulatory dimensions of writing. The present study was intended to build upon and extend the successes of
Bruning and his colleagues, specifically the use of linguistics and monitoring functions in writing to create a new writing self-efficacy scale that might be used to measure individuals’ writing quality.
The Present Study
The present study was designed to explicitly examine a category of writing self- efficacy not previously explored—metalinguistic self-efficacy for writing—and its relationship to writing quality. It was intended to extend prior research in two major ways. First, items were developed to create the Metalinguistic Self-Efficacy Scale, a scale that comprises metalinguistic abilities thought necessary for writing success. These items
(see Table 2.1) provide the potential for more explicitly describing important language- related activities occurring during writing. Currently, the prominent models of writing
(e.g., Hayes, 1996) only generally describe the writing process. Second, this study was intended to extend research on self-efficacy for writing, which mostly has been focused on relationships between self-efficacy for writing generally and writing outcomes. The current study provided an opportunity to more specifically examine writing process- product relationships, specifically those related to language-related decisions of writers, particularly in the realm of linguistics.
This study measured participants’ metalinguistic self-efficacy and used it as a proxy for rating the quality of their writing on an essay. Confidence for metalinguistic
46
Table 2.1: The Metalinguistic Self-Efficacy Scale Please read the tasks below and consider how successful you can be at them related to writing academic papers. Rate your confidence as a percentage using a scale of 0% to 100%. Zero percent equals “does not represent me at all,” 100% equals “represents me exactly. You can write ANY number in the 0-100% range that best describes you.
I can tell . . . 1. when to add prefixes and suffixes to words 2. when to use punctuation 3. if I understand my audience’s perspective on my topic 4. if I have provided good evidence for my argument 5. when to use singular and plural words 6. if my writing has the information my audience needs 7. when I should look for a better word 8. when my paragraph structures will impact my reader 9. which of two words better matches my meaning 10. if my words will impact my reader 11. when to blend multiple ideas into one sentence 12. if my grammar supports my writing 13. when my grammar in wrong 14. when to use spelling strategies I have learned 15. when I have formed a strong argument 16. when to use ‘this’ and ‘that’ to refer to other words 17. when to choose different sentence structures 18. when my argument needs to be clearer 19. if I have used the best argument structure 20. if I am making typing errors 21. when to apply spelling rules 22. if my pronouns refer clearly to my nouns 23. when to break words into syllables 24. if I have corrected all my spelling 25. when my words express my ideas exactly 26. if my written argument reaches my writing goals 27. when to use new words for a previous word 28. when to use short and long sentences for effect 29. when to apply grammar rules 30. if my writing is interesting and engaging 31. when to incorporate new ideas into my writing 32. if I have included enough ideas in my writing 33. when I should add more descriptive words 34. if my writing is convincing 35. if my audience will follow my organization
47 ability was compared to ratings of the quality of academic writing assignment to determine if metalinguistic confidence predicted participants’ writing quality. As the review in the preceding section shows, this was a new approach to writing self-efficacy, and one that had the potential to better specify the translation processes in writing.
Hypotheses
Three hypotheses were tested, based on the research questions presented in
Chapter 1.
1. Will a measure of writing self-efficacy based on metalinguistic theory be best
described by a single factor or by multiple factors?
a. H0: In the population under study, responses to the Metalinguistic Self-
Efficacy Scale will yield a single factor.
b. H1: If the null is rejected, we expect that there is a multi-factor structure
for metalinguistics.
c. H3: If the null is not rejected, we conclude there is insufficient evidence to
accept that there is a multi-factor structure for metalinguistics.
d. It was predicted that a five-factor structure would emerge. The items I
developed for the efficacy scale were deemed to be sensitive and well
specified to the degree that they would tap the metalinguistic constructs
they were intended to. As indicated above, the five dimensions clearly are
tied to previous research in writing self-efficacy and are consonant with
current writing models, but they do so with more precision and a more
robust theoretical foundation than current writing models.
2. Does metalinguistic self-efficacy relate generally to overall writing quality? 48
a. H0: In the population under study, metalinguistic self-efficacy will bear no
relationship to writing quality.
b. H1: If the null is rejected, we accept that metalinguistic factors do relate to
writing quality.
c. H3: If the null is not rejected, we conclude that there is insufficient
evidence to accept that metalinguistic factors relate to writing quality.
d. It was predicted that metalinguistics would relate to writing quality. The
items used in this study were developed based on a lengthy tradition of
linguistic and metalinguistic research. This research provides the
theoretical linguistic underpinning needed to better specify cognitive
writing models. Because writing is a linguistic activity, such a
specification was expected to relate to writing quality.
3. If there are separate factors for writing-related metalinguistic self-efficacy, are all
equally related to general and specific writing outcomes?
a. H0: In the population under study, metalinguistic factors relate equally to
writing quality.
b. H1: If the null is rejected, we expect that some metalinguistic factors relate
more strongly than others to writing quality.
c. H3: If the null is not rejected, we conclude there is insufficient evidence
that some metalinguistic factors relate more strongly than others to writing
quality.
d. It was predicted that metapragmatics and metatext would contribute to
writing quality in this study. While it was expected that individual 49 differences in all of the dimensions identified would be found, it was recognized that the population under study likely possessed relatively high ability for lower-level metalinguistic skills due to their expertise with the writing process. It was expected that there would be greater variability in higher-level metalinguistic skills, however, given the difficulty even skilled writers show with higher-order writing skills.
50
CHAPTER 3
Method
The purpose of this study was to examine whether metalinguistic self-efficacy is related to academic writing quality. The study was aimed generally at better understanding the translation process during writing and its possible impacts on writing outcomes. Metalinguistic self-efficacy is described along five dimensions: metaphonology, metasemantics, metasyntactics, metapragmatics, and metatext.
Interdisciplinary research, both cognitive and linguistic, suggests that together these dimensions represent the intentional monitoring of the declarative and procedural knowledge needed for successful translation of conceptual knowledge into writing. Self- efficacy theory suggests that confidence for carrying out metalinguistic tasks during writing can be used as an indirect measure of the metalinguistic tasks being performed in the translation stage of the Hayes (1996) writing model.
In the next sections the methods for the current study are laid out. First the participants and setting are described followed by a description of the procedures, measures and instruments, and research design. Briefly, undergraduate students were asked to complete, in order, a demographic and academic performance survey (see
Appendix A), complete the Metalinguistic Self-Efficacy Scale (MLSE—see Appendix
B), read a news story about teachers boycotting standardized testing (see Appendix C), and write an academic-level letter to the teachers’ superintendent based on their reactions to the news story (see Appendix D). Two rubrics were used for rating the writing assignment (see Appendix E and F). The script used for recruiting participants (see 51
Appendix G) is also included in the appendices. The data collected were analyzed using exploratory factor analysis, correlational analysis, and regression analysis.
Participants and Setting
Participants were 114 undergraduate students enrolled in undergraduate educational psychology courses at a Midwestern university. All students aged 19 or older and enrolled in the undergraduate educational psychology classes were invited to participate in the study, although they were not required to do so. The educational psychology instructors gave students choosing not to participate an alternate writing assignment. Participants included freshman, sophomores, juniors, and seniors, with most participants being sophomores and juniors. All participants were the age of legal consent—19 years old or older—and were predominately pre-education and education majors. Furthermore, participants were mainly female, and Caucasian.
The participant pool was expected to be well suited for examining this study’s research questions and hypotheses. As adults, the participants should have been exposed to all levels of metalinguistic ability, both in their writing and in academic discourse
(Camps & Milian, 2000; Gombert, 1992; Karmiloff-Smith, 1996; Myhill, 2012), though their metalinguistic abilities and self-confidence should still have varied individually
(Myhill, 2012; Fallahi et al.). Furthermore, as college students, participants would likely have had significant opportunities to write academic papers and receive feedback on their writing. They therefore should have had well-formed conceptions of their self-efficacy for academic writing and had the ability to make judgments on the self-efficacy scale reflective of their true ability. Moreover, this population has been understudied in 52 metalinguistic research (Alamargot & Fayol, 2009, Myhill, 2009), which offered an opportunity to better understand linguistically related writing processes in this age group.
Participants completed the study in a quiet college computer classroom reserved for this purpose. They were provided with all materials for recording their answers, including a computer word processor and a # 2 pencil. The metalinguistic self-efficacy instrument was titled Academic Writing Appraisal Inventory, and, to reduce response bias, participants were assured that responses would be anonymous (Bandura, 2006).
Measures and Instruments
Seven documents and instruments were used during the study. All may be found in the appendixes. The first was a brief survey (Appendix A) that included demographic information (year in school, race, and gender), and two questions related to participants’ academic and writing success. The second, the Metalinguistic Self-Efficacy instrument
(Appendix B) contained 35 items. A detailed explication and rationale for and construction of this instrument is included in the next section, Factor Items for the Self-
Efficacy Scale: Constructing the Metalinguistic Self-Efficacy Scale. The next instrument was a reading assignment (Appendix C), taken from the newspaper The Washington
Times in Seattle, WA. Participants read the newspaper material in preparation for completing their writing assignment. The fourth instrument was the writing assignment
(Appendix D), which was used to evaluate participants’ academic writing quality along with their self-reported average grade they receive on a college paper. The participants wrote a single draft, and this first draft was used for evaluation purposes. Participants wrote a polemical letter to the Superintendent of the Seattle school district. The topic was highly controversial in nature, teachers boycotting a mandated standardized examination. 53
It was anticipated this topic would engage participants and encourage them to write. The fifth and sixth instruments were rubrics that were used to rate participants’ writing
(Appendixes E and F). The five metalinguistic dimensions were assessed in the letters participants wrote during the study using criteria drawn from the metalinguistic dimensions. These criteria were used to develop two rating rubrics. The first specified analytic criteria for each metalinguistic dimension. For example, under metaphonology, cadence is given as an example of sound being used to create interest or meaning in writing. The second rubric was also analytical but identified holistic rating levels (1, 3, and 5, 5 being excellent). It combined all dimensions at each rating level in a more descriptive format to aid holistic rating of the participants’ writing. The seventh instrument was the introductory script that was used for recruiting participants (Appendix
G). It (a) introduced the researcher, (b) described the importance of the study and its possible benefits, and (c) provided a statement that participation was voluntary and would not affect students’ grade or relationship with their instructor or with the University.
Constructing the Metalinguistic Self-Efficacy Scale: The Five Metalinguistic Dimensions and Their Items
The five metalinguistic dimensions are presented below including the task items designed to tap each metalinguistic dimension. The task items were not weighted in any way—all were assumed to have the potential to contribute equally to participants’ writing quality. Each item begins with “I can tell,” as
Bandura (2006) suggests. This phrase is followed by one of a few words: when, if, or which. The purpose of these words was to encourage participants to reflect on their intentional monitoring during the writing process. The scale used was 0%- 54
100%, where 0% equals “Does not represent me at all” and 100% equals
“Represents me exactly” (Bandura, 2006). These items are representative of the activities that occur during translation, as reviewed in Chapter 2.
Metaphonological ability, the ability to control the sounds of language, is represented differently in writing than it is in oral communication or during reading
(Camps & Milian, 2000). As in oral communication, this ability involves phonemic awareness, but it also involves spelling and is affected by the medium being used to write. For the first ability, phonemic awareness, I used three items to tap participants’ confidence: (1) I can tell when to make singular and plural words; (2) I can tell when to add prefixes and suffixes to words; and (3) I can tell when to break words into syllables to spell them. The second ability is an awareness of correct spelling, which is fundamental to academic writing quality. Three items were used to measure participants’ confidence for this ability: (1) I can tell when I need to apply spelling rules; (2) I can tell if I have corrected all my spelling; and (3) I can tell when to use spelling strategies I have learned. The last ability, awareness of how to manipulate writing tools external to the writer might include pencils and word processors for transcribing (Kellogg, 1994;
McCutchen, 2012). A single item was used to measure participants’ confidence for this ability: I can tell if I am making typing errors. Knowledge of having made a typing error is evidence of monitoring activity.
Metasyntactic ability, as stated earlier, is described here as grammar and syntax.
Grammar comprises rules that govern the proper use of language, and four items were used to tap this dimension: (1) I can tell when to apply grammar rules; (2) I can tell when my grammar in wrong (3) I can tell if my grammar supports my writing; and (4) I can tell 55 when to use punctuation. Syntax is the structure, including at times grammatical rules, that is the form through which writers’ communicate their message. Three items tapped this dimension: (1) I can tell when to blend multiple ideas into one sentence; (2) I can tell when to use short and long sentences for effect; and (3) I can tell when to choose different sentence structures.
Metasemantic ability comprises both word choice and semantic activity.
Word Choice ability includes the knowledge that words are words and the skill to select words intentionally. Three items tapped this dimension: (1) I can tell when I should add more descriptive words; (2) I can tell when my words express my ideas exactly; and (3) I can tell if my words will impact my readers. Semantic ability reflects knowledge of the concepts words point to, both connotatively and denotatively, and how to manipulate words or longer signifying elements intentionally with the purpose of guiding the reader to a shared understanding of the intended conceptual space, that is, meaning. This is the equivalent of forming and informing propositional content. Two items tapped this dimension: (1) I can tell if I have included enough ideas in my writing; and (2) I can tell when to incorporate new ideas into my writing. Reflecting the unified nature of word choice and semantic ability, there are items that cross the two boundaries. Two items tapped the singularity of this cross-dimension: (1) I can tell which of two words better matches my meaning; and (2) I can tell when I should look for a better word.
Metapragmatic ability is the intentional monitoring of the context that contributes to meaning, for example, genre, rhetorical options, and audience characteristics. It includes audience perspective taking, recognizing the clarity of one’s argument, and recognizing the quality of one’s rhetorical choices. Audience perspective 56 taking is the activity of consciously identifying the intended audience, how this audience thinks, feels, and what they know about the writer’s topic. Two items were used to tap this dimension: (1) I can tell if my writing has the information my audience needs; and (2)
I can tell if I understand my audience’s perspective on my topic. Recognizing the clarity of one’s argument is the activity that utilizes the internal editor at the metapragmatic level. Three items tapped this dimension: (1) I can tell when my argument needs to be clearer; (2) I can tell when I have formed a strong argument; and (3) I can tell if I have provided good evidence for my argument. Recognizing the quality of one’s rhetorical goals is the activity where the writer selects an appropriate genre and argument format to present the writing topic. Two items tapped this dimension: (1) I can tell if I have used the best argument structure; and (2) I can tell if my written argument reaches my writing goals.
Metatextual ability includes the constructs cohesion and coherence. Cohesion refers to linguistic related, repetitive textual devices; they are words that allow writers to vary their writing for interest or other intentions. Three items tapped this dimension: (1) I can tell when using new words for a previous word makes sense; (2) I can tell if my pronouns refer clearly to my nouns; and (3) I can tell when to use ‘this’ and ‘that’ to refer to other words. Coherence refers to the clarity of shared meaning represented in the text in part or in whole. Four items tapped this dimension: (1) I can tell when my paragraph structures will impact my reader; (2) I can tell if my writing is convincing; (3) I can tell if my audience will follow my organization; and (4) I can tell if my writing is interesting and engaging.
57
Procedure
The researcher visited four classrooms of undergraduate educational psychology classes to recruit participants. The researcher was not an instructor for any of the educational psychology courses, and when the researcher arrived the instructor left the room to eliminate any possibility of coercion during the recruitment process. Instructors announced to the class one week ahead that the researcher would visit to present a research opportunity. Once the instructor had left the room, the researcher read a script
(see Appendix G) that explained the research purpose, activities, potential benefits to participants, potential benefits to writing research, and students’ rights and eligibility.
The script finished with a call for questions and concerns. There were a few questions on the meaning of metalinguistics and one question related to privacy. Participants were given a list of available times to complete the study that they could choose from to best meet their schedules. No prior sign-up was necessary; participants were allowed to arrive and complete the study as they chose. The instructors did not return to the class until the recruiting activity was finished. Recruiting time was approximately 10 minutes.
The day participants arrived for the study, they signed an Informed Consent form and completed a demographic survey (see Appendix A). Next, participants completed the
Metalinguistic Self-Efficacy Scale (MLSE—see Appendix B). Following the survey, participants read a short informational piece taken from a newspaper article about teachers boycotting a standardized examination (see Appendix C). The topic is pertinent to preservice teachers and draws strong sentiment from the field of education generally.
Participants then wrote an academic-style letter to the Supervisor of the school district mentioned in the article (see Appendix D). It was expected that the topic would engage 58 participants and encourage them to write thoughtfully. Participants were given rein to think about, plan, organize, and write their papers. During the study, participants had available to them paper and pencils in addition to Microsoft Word. The paper and pencil were for planning, though no one took advantage of it; participants typed their letters in
Microsoft Word on a computer and saved their papers on the computer desktop. The study lasted approximately 20 to 30 minutes.
To ensure anonymity, all of the study materials were handed out in a manila envelope with a unique number written on the front and the instructions “Open.” The
Informed Consent form was signed and set aside so it could be turned in separately from the other materials, which had no identifiers on them. The Word document was saved with the same unique number as was printed on the front of the manila envelope. Once papers were saved, participants returned the Informed Consent form in one box and the manila envelope—containing the unidentified, completed demographic sheet and
MSLE—in another box. The researcher immediately went to each computer and (a) saved the participant’s essay to a hard drive, and then (b) permanently erased the participant’s electronic essay from the desktop.
Research Design
The current study utilized a correlational design, namely a prediction correlational design. The researcher in this study sought to anticipate outcomes on a writing sample and self-reported average grade on a college paper, or criterion variables, using the
MLSE scale and its items as predictors. The purpose of the prediction correlational design was to positively identify metalinguistic items that predict writing quality and are appropriate for measuring “the degree of association (or relationship) between two or 59 more variables or sets of scores” (p. 361, Creswell, 2002). The primary independent variable was the MLSE (see Appendix B), and the dependent variables were the ratings on the completed essays and self-reported average grade on a college paper. Furthermore, because there were anticipated to be multiple predictor variables, regression analysis was used, as described below in the Data Analysis section.
Data Analysis
Exploratory factor analysis was used to address Research Question 1: Will a measure of writing self-efficacy based on metalinguistic theory be best described by a single factor or multiple factors? Successful implementation of factor analysis relies on a careful choice of variables and participants to ensure high correlation among the variables
(Nunnally & Bernstein, 1994). A medium sample size of 115 was used to ensure variance did not reflect sampling error, (Hinton, 2004, Nunnally & Bernstein, 1994). Exploratory factor analysis in the current study was superior to a confirmatory factor analysis, which is commonly used when the investigator is working from existing theory and has identified latent variables a priori. In reality, investigators using exploratory factor analysis approach their research design intuitively with factor structures already in mind
(Nunnally & Bernstein, 1994). Furthermore, although the current study utilized two well- researched theoretical bases—metalinguistics and self-efficacy—metalinguistic self- efficacy had yet to be studied systematically, and therefore exploratory factor analysis was chosen for analysis.
Reliability analyses of the items on the MLSE scale were measured using
Cronbach’s alpha. Cronbach’s alpha is preferred because it is based on the average inter- item correlation of items in the instrument (Hinton, 2004). If no error were present, 60 reliability would be equal to 1.00, representing perfect correlation between the items and perfect reliability. Perfect reliability was not expected, however, but based on prior self- efficacy for writing research (e.g., Bruning, et al., 2013) and the methods used to develop the items based on metalinguistics theory, it was expected that Cronbach’s alpha would be sufficiently high to permit the use of the MLSE.
Correlational analysis was used to address Research Question 2: Does metalinguistic self-efficacy relate generally to judgments of overall writing quality?
Decades of research in linguistics, metalinguistics, and self-efficacy suggest that dimensions of the MLSE would positively predict similar dimensions on the academic writing assignment participants completed (see Appendix D) and also predict self- reported average grade on a college paper. The MLSE was the primary predictor variable
(see Appendix B). The predicted, or criterion variables, were ratings of the quality of student essays (see Appendix D) and self-reported average grade on a college paper.
Finally, multiple linear regression analysis was used to address Research
Question 3: If there are separate factors for writing-related metalinguistic self-efficacy, are all equally related to general and specific writing outcomes? The purpose of the regression analysis was to determine if one or more predictor variables might explain a disproportionate amount of variance on the criterion variables (Pedhazur, 1997). Writing quality was rated using two scales developed especially for identifying dimensions of metalinguistic ability (see Appendixes D and E), with a holistic, single measure of writing quality being given to each essay.
During data-analysis, SPSS (version 21, 2013) was used for performing the exploratory factor analysis, correlation analysis, and regression analysis. The writing 61 samples were evaluated by two raters—the researcher and a second reader—who were blind to the participants and their MLSE responses. The researcher trained the second reader in rating the papers. Two rubrics (Appendix E and F) were used during training and referred to as needed during scoring. The first rubric was an operationalization of metalinguistics (e.g., cadence would represent an intentional use of phonology). Using the first rubric, raters mentally noted evidence related to each dimension for each paper and assigned an overall score for each dimension at the end of reading the essay. Raters used these analytical scores in conjunction with the second rubric, which listed identifiers for three levels of performance (1, 3, and 5, with 5 being Excellent). The readers assigned a holistic score to each essay based on the second rubric, and using the first rubric to help generate and support the holistic rating. The readers read a sample of the papers, comparing scores and coming to agreement on differences when they could. The scores of the two readers were averaged if they were no more than one scoring point from each other. This occurred 4 times during the rating process. If they were more than one scoring point away, a third rater, trained by the researcher, was to make a rating that served as the tiebreaker. Seeking a third reader was not necessary as the raters did not disagree by more than one scoring point on any paper. From this process, a reader reliability analysis was conducted. Based on previous experience (Dempsey, PytlikZillig, & Bruning, 2009), it was anticipated that reader reliability would be ∝ = .90 or higher. 62
CHAPTER 4
Results
The Metalinguistic Self-Efficacy Scale (MLSE) was administered to 115 participants, all college students enrolled in undergraduate educational psychology courses. One participant failed to save his essay properly on the computer; therefore, the data of 114 participants is presented in the results below. Descriptive statistics are presented first followed by results for the three hypotheses tested. Hypothesis 1 results are presented first: Will a measure of writing self-efficacy based on metalinguistic theory be best described by a single factor or by multiple factors? Next, Hypothesis 2 is addressed: Does metalinguistic self-efficacy bear a relationship to writing quality?
Finally, Hypothesis 3 is discussed: If there are separate factors for writing-related metalinguistic self-efficacy, are all equally related to general and specific writing outcomes?
Ratings on the participants’ essays were well dispersed across the 5-point scale used to score the essays. Participants were asked to write a response to a teacher boycott of standardized exams. The audience was the superintendent of the school system where the teachers work. Two raters were used initially to establish inter-rater reliability following procedures used by Dempsey et al. (2009). Twenty-eight participant essays, representing 20% of the total essays written, were rated independently by two readers who were blind to participants and their responses to the MLSE. One rater was the researcher, the other a psychology faculty member with eleven years experience grading college student papers. Training the second rater involved familiarizing her with the study and the writing topic, familiarizing her with the two rubrics and how they relate to 63 writing, and having her practice independently on five papers to ensure rating accuracy and consistency. Inter-rater reliability was computed using inter-class correlation: The correlation between the two raters was r = 0.96. The high inter-rater reliability on the sample of essays suggested that the researcher was rating essays consistently and so the researcher finished rating the remaining essays alone. All essays were first rated analytically for the five metalinguistic constructs. Next, based on the analytic scoring, an overall, holistic rating was assigned. The holistic ratings were used to conduct the analysis. The mean for essay ratings was 3.35 on a 5-point scale (SD = 1.11).
It is worth noting that the essays generally showed high levels of participant engagement as shown by average word count and the emotion often exhibited toward the subject matter, suggesting participants were motivated to write to the topic. Word counts were calculated as an ancillary statistic that was not utilized in the main analyses. The average word count of the essays was 351.04 (SD = 145.86) as calculated using the software Linguistic Inquiry and Word Count (version 2007; Pennebaker, Booth, &
Francis, 2007). Each participant on average produced 1½ pages of text, double-spaced, for the topic of teacher boycotts of standardized testing. Remembering that the entire study lasted less than 30 minutes, the length of participants’ essays was judged to be high for a research study of this type and even for academic writing assignments generally
(Fallahi et al., 2006). The majority of the essays showed passion for the topic, irrespective of the ratings received on individual essays. The following excerpt, from an essay rated “3,” provides an excellent example of the emotions, and possible motivation, some participants displayed when writing their responses:
64
Participant #74:
“I believe that this boycott is a forceful and great idea for the teachers to
be doing to take action. As a student in the College of Education here at
the [state university] I have taken a School and Society course. In this
course we have discussed the No Child Left Behind act that was put into
place by the Bush Administration. We discussed how it based too much of
the students curriculum on standardized testing, to the point that teachers
are not even able to teach students basic life skills because they are more
worried about teaching the children to the test.
. . . Take the time to thank your children’s teachers. Take the time to ask
them questions about how they feel about certain issues in the field of
education. Take the time to get their views on the situation instead of
making an assumption.”
Participant #74 received average to high marks in Conventional categories, but only average to low marks in Ideational categories, (e.g., the paper did not fully address audience needs for information, did not fully recognize the audience’s perspective, and lacked cohesion and coherence). This participant’s paper, however, was not average in length. This participant wrote an essay 4 pages double-spaced and 1035 words long, which might be taken as a measure of the motivation this participant felt for writing to the topic.
The means and standard deviations, along with information on skewness of the distribution of scores are presented in Table 4.1 below for (1) all 35 Metalinguistic Self- 65
Efficacy items, (2) ratings on the essays participants wrote after completing the MSLE, and (3) participants’ self-reported average grade on their academic papers.
Table 4.1 Means, Standard Deviations, and Skewness for Responses to the Metalinguistic Self- Efficacy Scale, Ratings on Essays, and Self-reported Average Grade on Academic Papers Standard Metalinguistic Self-Efficacy Scale Items Mean Skewness Deviation when to add prefixes and suffixes to words 89.97 14.21 -2.41 when to use punctuation 86.62 13.00 -1.17 if I understand my audience’s perspective on my topic 82.05 14.18 -1.77 if I have provided good evidence for my argument 85.32 10.81 -1.04 when to use singular and plural words 93.07 12.74 -2.82 if my writing has the information my audience needs 85.89 11.00 -1.09 when I should look for a better word 83.94 17.09 -1.79 when my paragraph structures will impact my reader 77.21 18.37 -1.15 which of two words better matches my meaning 82.61 17.40 -1.90 if my words will impact my reader 78.75 16.59 -0.99 when to blend multiple ideas into one sentence 78.25 16.62 -1.09 if my grammar supports my writing 81.31 17.24 -1.26 when my grammar is wrong 82.62 17.92 -1.41 when to use spelling strategies I have learned 80.40 20.37 -1.64 when I have formed a strong argument 84.98 11.98 -1.10 when to use ‘this’ and ‘that’ to refer to other words 87.72 13.00 -1.67 when to choose different sentence structures 77.73 19.72 -1.49 when my argument needs to be clearer 79.42 15.70 -1.41 if I have used the best argument structure 75.36 16.49 -1.10 if I am making typing errors 88.11 14.25 -1.61 when to apply spelling rules 85.23 16.87 -1.86 if my pronouns refer clearly to my nouns 89.79 15.05 -3.32 when to break words into syllables 83.71 17.40 -1.89 if I have corrected all my spelling 87.54 16.29 -2.45 when my words express my ideas exactly 83.12 18.52 -1.89 if my written argument reaches my writing goals 81.84 17.13 -1.89 when to use new words for a previous word 84.28 14.09 -1.10 when to use short and long sentences for effect 78.72 18.24 -1.06 when to apply grammar rules 80.96 19.29 -1.57 if my writing is interesting and engaging 82.45 16.48 -1.34 when to incorporate new ideas into my writing 80.62 16.59 -1.23 if I have included enough ideas in my writing 80.54 16.00 -1.24 when I should add more descriptive words 78.98 19.39 -1.41 if my writing is convincing 80.62 15.88 -1.24 if my audience will follow my organization 80.10 16.27 -1.43 Ratings on Participants’ Essays, Self-Reported Average Standard Means Skewness Grade on an Academic Paper Deviation Rating on participants’ essays 3.35 1.11 -0.03 Self-reported average grade on academic papers 10.89 1.20 -4.70
66
At times there was a wide range in responses to self-efficacy items (e.g., 0 to 100 for I can add descriptive words). Means for all items, however, tended to be high, and negative skewness found for all the items indicated clustering of ratings around the higher end of the self-efficacy scales. Ratings on participants’ essays were more normally distributed, with a mean rating of 3.35 on a 5-point scale (SD =1.11, skewness = -0.03; see Table 4.1). The skewness index describes a distribution that is more clustered around the mean, indicating a more normal distribution for this variable. Self-reported average grades on college papers revealed a high mean, A- or 10.89 (SD = 1.20) on a 12-point scale. Self-reported average grades on college papers showed a negatively skewed distribution equal to -4.70, indicating that participants received generally high marks on their academic papers.
Factor analysis (SPSS 21, 2013) was used to clarify the data set in order to address Hypothesis 1: Will a measure of writing self-efficacy based on metalinguistic theory be best described by a single factor or by multiple factors? KMO Bartlett’s test of sphericity was used to measure sampling adequacy, which was significant at p < .01, indicating the sample size for the factor analysis was adequate. An initial, non-rotated analysis revealed five factors with eigenvalues all above 1. Total variance explained by the five factors was 72.52%. However, the scree plot indicated that at best only 2 or 3 factors were present. Zwick and Velicer (1986) have strongly advised using Horn’s
(1965) parallel analysis, a method for determining the number of factors a researcher should retain or extract from a factor analysis. The analysis utilizes a program, such as
Parallel Analysis Engine (Patil, Surendra, Sanjay, & Donovan, 2007) to create a random dataset with the same number variables as the original dataset. A correlation matrix is 67 computed from the randomly generated dataset followed by computing eigenvalues of the correlation matrix. When the eigenvalues from the random data are larger than the eigenvalues from the factor analysis it is assumed that factors are mostly random noise and the analysis stops. A parallel analysis was calculated, which generated a line that neatly bisected the second factor (see Figure 4.1), indicating that the researcher should retain just two factors.
The factor analysis was rerun in SPSS forcing the data into two factors (as advised by NEAR Center personnel) and using a Varimax rotation. After cross-loading items were removed, 24 of the original 35 items remained, 13 items in Factor 1 and 11 in
Factor 2. Final loadings for the 24 items can be found in Table 4.2. Factor 1 accounted for 35.11% of the variance, Factor 2 accounted for 28.93% of the variance, and total variance explained was 64.03%. The two factors appeared to be theoretically distinct from one another, with each showing considerable theoretical cohesion.
Figure 4.1: Parallel analysis of factor structure suggesting a two-factor model 68
The null hypothesis for Hypothesis 1 therefore was rejected: metalinguistic self-efficacy is best described by multiple factors, although not by the five factors that were originally proposed.
This study and its instrumentation were based upon a five-construct metalinguistic structure. While the two factors found contained items representing all of the constructs, it should be noted that the items sorted themselves more parsimoniously during factor analysis than was originally presumed, and that the two factors required broader labels than metaphonology, metasyntax, metasemantics, metapragmatics, and metatext. Factor 1
(see Table 4.2) was made up of items tapping skills regularly assumed to be higher-order writing abilities. In this study these included items related to metapragmatics and metatext. Factor 2 was made up of lower-order writing abilities, in this case metaphonology and metasyntax. Metasemantics bridges both factors. This study was designed, in part, to build upon work of Bruning et al. (2013), who in addition to a more general self-regulatory dimension of writing self-efficacy, likewise found two linguistic factors using their self-efficacy for writing instrument. Bruning et al. (2013) provide the terms Ideation and Conventions for these linguistically-related factors, which generally identify with higher-order and lower-order writing abilities respectively, and these terms will be adopted here. From this point, Ideation will be used to refer to the present Factor
1, Conventions to Factor 2.
Mean ratings of self-efficacy were 82.13 (SD = 12.05) for Ideation and 84.90 (SD
= 13.18) for Conventions. Reliabilities for these two sets of items were high with r = 0.95 for the 13 items comprising Ideation and r = 0.93 for the 11 items in Conventions.
69
Table 4.2 Final Factor Loadings for a Two-Factor Model of Metalinguistic Self-Efficacy Factor and item Factor loading Ideation I can tell . . . when to incorporate new ideas into my writing (item 31) 0.87 if my writing is convincing (item 34) 0.83 if my writing is interesting and engaging (item 30) 0.81 if my audience will follow my organization (item 35) 0.82 if I have included enough ideas in my writing (item 32) 0.79 when I have formed a strong argument (item 15) 0.72 if my writing has the information my audience needs (item 6) 0.70 when my paragraph structures will impact my reader (item 8) 0.70 when to blend multiple ideas into one sentence (item 11) 0.67 if my words will impact my readers (item10) 0.66 when using new words for previous words makes sense (item 27) 0.66 if I have provided good evidence for my argument (item 4) 0.63 when to use ‘this’ and ‘that’ to refer to other words (item 16) 0.62 Conventions I can tell . . . which of two words better matches my meaning (item 9) 0.74 if my pronouns refer clearly to my nouns (item 22) 0.74 when to apply spelling rules (item 21) 0.73 when to apply grammar rules (item 29) 0.73 when to break words into syllables (item 23) 0.72 when to use singular and plural words (item 5) 0.71 when to add prefixes and suffixes to words (item 1) 0.70 if my grammar supports my writing (item 12) 0.69 when to use spelling strategies I have learned (item 14) 0.68 when I should look for a better word (item7) 0.66 if my grammar is wrong (item 13) 0.55
The correlational analysis addressed Hypothesis 2: Does metalinguistic self- efficacy relate generally to overall writing quality? Ideation and Conventions correlated highly with one another, r = 0.79, p < .01. It is perhaps not surprising that two factors so integral to the writing process should correlate highly. There were also significant correlations between ratings on the essays and self-reported average grade on college papers, both writing quality indicators (r = .41, p < .01). Ideation yielded significant correlations with self-reported grade on college paper only, r = 0.32, p < .01. Conventions, however, was significantly correlated with both ratings on essays, r = 0.23, p < .05 and with average paper grade, r = 0.25, p < .01 (see Table 4.3). The correlations between Conventions and the writing quality and academic indicators are indicative of the results obtained by prior writing self-efficacy researchers (Bruning et al, 2013; Pajares, 2007; Zimmerman & Bandura, 1994). These results offer evidence that there is a 70 relationship between metalinguistic constructs and writing quality, therefore the null hypothesis for Hypothesis 2 is rejected.
Table 4.3 Correlations for Ideation, Conventions, Ratings, and Self-Reported Average Grade on Academic Papers Measure Conventions Ratings Self-Reported Average Grade on Academic Paper Ideation 0.77** 0.17 0.32**
Conventions 0.23* 0.25**
Ratings 0.41**
*p < .05. **p < .01
Multiple regression analysis was used to address Hypotheses 3: If there are separate factors for writing-related metalinguistic self-efficacy, are all equally related to general and specific writing outcomes? The first regression model tested the relationship between Ideation and Conventions and ratings on the participants’ essays. The regression model approached significance, R2 = .05, F (2, 111) = 3.05, p = .05. The standardized coefficients for Ideation was β = -0.1, p = .92 and for Conventions β = 1.66, p = .10.
Conventions here was marginally significant p = .10, suggesting that Conventions points in the direction of predicting more of the variance than Ideation for ratings on participants’ essays, though, again, the overall regression model only approached significance.
The regression model using self-reported average college paper grade as the criterion variable and Ideation and Conventions as predictor variables was significant, R2
= .10, F (2, 111) = 6.32, p < .01. The standardized coefficient for Ideation was β = 0.31, p
= .03 and for Conventions β = 1.66, p = .10. Ideation was significant at p < .05 indicating that that this factor predicts more of the variance for self-reported average grade on 71 college paper. This is the converse of the regression results for ratings on participants’ essays.
Taken together, the regression analyses suggest that the two predictor variables,
Ideation and Conventions, do not equally account for the variance of the criterion variables, and the null hypothesis for Hypothesis 3 is rejected. The two regression analyses showed Conventions explaining more of the variance for the ratings on participants’ essays, and Ideation explaining a significant amount of the variance for the self-reported average college paper grade. These differences will be explored further in
Chapter 5. 72
CHAPTER 5
Discussion
The purpose of this study was to examine the relationship between a set of metalinguistic skills potentially used by college students during writing to actual quality of the writing they produce. In this study, based on consistent relationships noted in prior research between self-efficacy in multiple domains and actual performance in those domains (Bandura, 1986, 1997; Bruning, et al., 2013), self- efficacy for executing metalinguistic skills during writing was used as an indirect measure of (i.e., a proxy for) actual use of metalinguistic skills during writing.
Two measures of writing performance were used: (1) ratings of student responses to a writing assignment, which was judged using rubrics developed from the five metalinguistic dimensions; and (2) self-reported average college paper grade.
Findings from this study were meant to aid our understanding of individuals’ control over linguistic processes occurring during translation, something arguably absent in current writing models, as well as providing information about the extent to which metalinguistic activities during writing affect writing quality. The study was deemed to have the potential to provide information about important metalinguistic activities occurring during translation and how they relate to the quality of writing that is produced.
This study addressed three research questions: (1) Will a measure of writing self- efficacy based on metalinguistic theory be best described by a single factor or multiple factors?; (2) Does metalinguistic self-efficacy relate generally to judgments of overall 73 writing quality?; and (3) If there are separate factors for writing-related metalinguistic self-efficacy, are all equally related to general and specific writing outcomes?
Regarding Research Question 1, Will a measure of writing self-efficacy based on metalinguistic theory be best described by a single factor or multiple factors?, two factors were found that accounted for a high proportion of variance in the Metalinguistic Self-Efficacy (MLSE) scale. It was anticipated, from a metalinguistic theoretical point of view, that as many as five factors would emerge (Gombert, 1992; Karmiloff-Smith, 1981, 1996; Myhill, 2012). The original, unrotated factor analysis did show five factors initially, but there were significant crossloadings and the scree plot and parallel analysis indicated that two factors only should be retained. Those two factors, Ideation and Conventions, appeared to be representative of higher- and lower-order metalinguistic abilities
(Bruning et al., 2013; Gombert, 1992; Karmiloff-Smith, 1996; Myhill, 2012;
Pajares, 2007). The significance of this finding is that participants (1) can identify personal self-efficacy for metalinguistic task abilities generally, and (2) self- efficacy for these abilities falls into categories that earlier have been shown to be associated with writing quality (Bruning et al, 2013; Pajares, 2007).
One obvious question is why two factors and not five, as originally presumed? Based on the factor analysis, and subsequent correlational and regression analyses, we can assume that the writing outcomes used in this study are generally related to metalinguistic self-efficacy. The two factors observed in the present data tap all five of the metalinguistic constructs described in Chapter
2. They also are consistent with multiple findings from previous research on 74 writing self-efficacy that have revealed a lower-order factor and a higher-order factor in writing self-efficacy (e.g., Bruning et al., 2013; Pajares, 2007;
McCutchen, 2011, 2012).
Still, there are only two factors, not five as metalinguistic theory would predict. There would seem to be at least three possible reasons why factor analysis did not yield more metalinguistic dimensions. First, it may be that items on the
MLSE were not sensitive enough to detect more than two factors, though the factors included new and more specifically linguistic items than previous research has utilized. If the MLSE is at fault for not yielding more factors, scale construction may be to blame (e.g., DeVellis, 2011). Writing scale items requires meeting several constraints concurrently: items must be brief; items must be written to avoid confusion and alternative interpretations; several items may be needed to fully capture the reality of the task under study; and most important, items must have face and construct validity. To help insure validity, items should be vetted with experts in the field (DeVellis, 2011). Not all of these constraints were met in the construction of the MLSE. For example, while the scale was shown to experts in self-efficacy, it was not passed to anyone with expertise in metalinguistics. Some items also may have lacked clarity (e.g., participants likely might have judged item 22—pronouns refer clearly to my nouns—as a grammatical item rather than the higher-order meta-text item it was meant to reflect). The fact that so many items crossloaded is another indication that items perhaps were not perceived as clearly as they should have been. Still, it is 75 important that two factors were observed, and these factors seem clearly to point to two general metalinguistic self-efficacy factors.
Second, the level of writing expertise of the participants (i.e., the nature of the participant group) may have been such that they did not distinguish among metalinguistic abilities at the fine-grained detail that the MLSE was designed to capture. Translation, as early writing research assumed (Alves, 2012), may in fact be an automatic process. Metalinguistic research also has, for the most part, undersampled adult populations (Alamargot & Fayol, 2009, Myhill, 2009).
Instead, as stated in Chapter 2, it has been focused more on developmental writing issues in children (Gombert, 1992; Karmiloff-Smith, 1981, 1996; Berninger,
1996). It may be that adult writers in college have sufficient expertise in metalinguistics that they simply do not distinguish between, say, spelling and punctuation when it comes to their metalinguistic ability.
Third, it is possible that the individual writing abilities used during the writing process (i.e., translation) are more holistic, more conflated, and therefore harder if not impossible to parse than was assumed at the start of the study, particularly with a college-aged participant group. Studies with children (Bereiter
& Scardamalia, 1987; Berninger & Swanson, 1994; McCutchen 2011, 2012) show that if they are struggling with issues such as transcribing, they still must grapple with higher-order metalinguistic abilities such as audience. This problem may implicate working memory and long-term working memory (Kellogg, 2001) more than it does metalinguistic ability. In fact, this study is based on the assumption that all five metalinguistic abilities are managed concurrently for 76 writers at any age, though some abilities may require more attention than others depending on individual ability. The question raised here is whether the MLSE is sufficiently broad and sensitive enough to capture individuals’ ability to marshal and apply the five metalinguistic abilities represented in the MLSE.
Regarding Research Question 2, Does metalinguistic self-efficacy relate generally to judgments of overall writing quality?, the correlation analysis showed relationships between the factors and both of the writing outcomes.
Ratings on the participants’ essays showed a positive correlation with
Conventions only, while both Ideation and Conventions showed positive relationships with self-reported average college paper grade. The relationship between self-reported average college paper grade and the two factors suggests a relationship between writing quality and metalinguistic self-efficacy, used in this study as a proxy for metalinguistic ability that may be predictive of writing ability.
That Ideation did not correlate with what is arguably the most important writing outcome for the study—objective ratings of the quality of participants’ essays—may suggest that the relationship between the two predictor variables was affected by participants’ involvement with the provocative topic to which they wrote—teachers boycotting standardized exams. Writing prompts that tap the interests of writers can affect the quality of writing outcomes even when writing ability is low (Benton, Corkill, Sharp, Downey, & Khramtsova, 1995). This is not to suggest that metalinguistic Ideational competence is neutral in its impact on writing when interest and motivation in the writing topic are high. I speculate 77 here, however, that Ideational and Conventional abilities vary in the degree that they support writers when addressing highly engaging topics. In other words, participants may not have required high levels of metalinguistic self-efficacy for
Ideation to produce quality writing on the topic to which they wrote. This might be particularly true given that the topic was not of the same academic genre that participants are used to writing in and to which they self-judged themselves on the
MLSE at the start of the study. We can speculate, based on the correlations observed between Ideation, Conventions, and rating’s on participants’ essays, that participants’ Conventional metalinguistic abilities may in some cases be all that are necessary for judging an essay’s quality, when the writer is highly motivated and engaged, as observed in Participant #74’s writing in shown in Chapter 4. That essay, with average to high Conventional marks and low to average Ideational scores, nevertheless can resonate with the reader.
It is also possible that the researcher’s ratings on participants’ essays were affected sub-consciously by emotionally powerful writing. In effect, the ability to resonate with one’s reader may cause a rater to overlook or forgive writing that does not exhibit qualities found on a writing assessment rubric. It is possible that the process used to rate the essays may have in some instances downplayed
Ideational qualities when the writing was otherwise emotionally moving, and these instances may have affected the correlation between Ideation and ratings on participants’ essays.
It is common, furthermore, for some writing researchers (Bereiter &
Scardamalia, 1987; Hayes 1996, McCutchen, 2011, 2012; Myhill, 2012) to 78 assume that higher order linguistic abilities mark the line between expert and less accomplished writers. In fact, such an assumption may help explain why Ideation and Conventions both correlate with self-reported average college paper grade: participants receive a great deal of feedback on their academic writing, a type of writing that values both Ideational and Conventional metalinguistic abilities.
Regarding Research Question 3, If there are separate factors for writing- related metalinguistic self-efficacy, are all equally related to general and specific writing outcomes?, this study found both equal and unequal relationships between the predictor variables, Ideation and Conventions, and the criterion variables, ratings on participants’ essays and self-reported college paper grades. The first regression analysis used ratings on the participants’ essays as the criterion variable and the two factors as the predictor variables. This analysis only approached significance. The second regression analysis used self-reported average college paper grade as the criterion variable and the two factors as the predictor variables. The relationship between the metalinguistic factors and average paper grade was significant at p < .01. The results from the regression analyses may indicate that some combination of genre (i.e., letter writing in the present study) and the possible increased motivation (due to the controversial topic) does not necessarily require the same Ideational effort or skill as another genre or topic might. Benton, et al. (1995), for example, using hierarchical regression, found an interaction between interest and grade on a writing task about baseball. They found that children with lower writing skills but high interest 79 in baseball could write essays superior to participants with higher writing ability but low interest in baseball.
Implications and Next Steps
This study suggests several possibilities for writing education. First, the presence of more than one metalinguistic factor tied to writing outcomes implies that writing is likely to be a process that interacts between lower-order and higher-order metalinguistic abilities that are within the purview of writers, not external to them.
Second, it may be that, as the differences between regression analyses suggest, genre and topic may play a role in how lower- and higher-order abilities interact to affect writing quality, as many writing educators and researchers have suggested (Calkins, 1994; Elbow, 1994; Hayes, 1996). Some writing tasks and topics may require different levels of Ideational or Conventional control to affect writing quality. It remains a question, though, if genre and topic play the penultimate roles given them by writing models —that is, being portrayed as constructs largely external to writers in traditional writing models. While it appears from this study that genre and rhetorical topic may play an important role, it is not clear that it drives the writing process as much as Hayes (1996) suggests.
More research in this area is warranted. For example, following the lead of
Benton et al. (1995), a hierarchical regression design might be utilized to test for interactions between interest generated by a writing topic and ratings of metalinguistic self-efficacy. The MLSE might be used to predict writing quality much as it was in this study. 80
Third, in line with previous writing self-efficacy research (Bruning et al.,
2013; Pajares, 2007; Zimmerman & Kitsantas, 2002), the current study demonstrates that writing self-efficacy is not unidimensional. This study confirms and extends earlier research in this area by identifying lower-order and higher- order factors and by having focused explicitly on the potential linguistic foundations of writing self-efficacy. For example, Pajares (2007) focused on just one higher-order construct, meta-text (e.g. paragraph structuring and topic sentences), while Bruning et al. (2013) focused on metasemantics with one factor tied to meta-text. The fact that both Pajares and Bruning et al. found results consistent with their theoretical approaches underlines the multidimensional nature of the writing process. The current study attempted to further specify the linguistic nature of the writing process by using a five-construct metalinguistic scale. This multidimensional approach, while not supported at the level of the five factors predicted in this study, did yield two clearly separate factors with an item set representing all five metalinguistic dimensions. We saw that the higher-order factor, Ideation included meta-pragmatics as well as meta-text items, while
Conventions, the lower-order factor, included meta-phonology and metasyntax items. At the same time, metasemantics was seen to bridge the two factors. It is possible, therefore, to view this study as making an important step toward better specifying the linguistic dimensions of the translation space found in writing models (e.g., Bereiter & Scardamalia, 1987; Hayes, 1996).
Finally, given the results from the present study, two “next steps” suggest themselves. First, the current MLSE should be reviewed and modified as 81 necessary. Most particularly, any adaptation of the current scale should be reviewed by experts in metalinguistics, as well as writing self-efficacy, and tested with target populations prior to conducting any future studies (DeVellis, 2011).
Second, an obvious next step forward from the current exploratory factor analyses is to run a new study with a larger sample size to permit confirmatory factor analysis. In the current study, we can assume that the exploratory analysis indeed yielded results that are of interest to writing research, but to be certain of their contribution to the theory of metalinguistic self-efficacy, a confirmatory factor analysis is unquestionably in order.
Limitations to the Current Study
The present study is subject to limitations that should be considered by future researchers. First, there appears to be little or no research in metalinguistic self-efficacy. It was for this reason that the current study utilized an exploratory factor analysis. The current study, therefore, should be seen as a start in the area of metalinguistic self-efficacy study. Furthermore, given that results from this study did not entirely match initial expectations, future research in this area should fully review the theory, instruments, methods, and procedures used in the current study.
Second, the population studied, college students, has unique characteristics related to writing that may have affected the results. Self-reported average grade on college papers were high, indicating that participants were generally skilled in writing, at least for academic essays, and participants generally showed high metalinguistic self-efficacy at the start of the study. These 82 characteristics suggest participants’ skills may be highly automatized and indicative of a high level of writing expertise. Future work in metalinguistic self- efficacy should utilize the Metalinguistic Self-Efficacy Scale, or a version of it, with a population of writers more diverse in writing abilities.
Finally, it is possible that the researcher was swayed by participants’ emotive writing, an example of researcher bias. Future researchers may eliminate this potential problem by employing a second rater for all participants’ essays instead of just a percentage. Additionally, future researchers may wish to amend the rubrics used in this study to include a rating for emotion and impression. This would create a new trait and help objectify it, possibly lessening its affect on the holistic score given to participants’ essays.
Conclusion
This study yielded important findings and also raised intriguing questions. It is significant that the Metalinguistic Self-Efficacy Scale supported previous findings of writing self-efficacy while pushing what we know of this field in new directions. This study adds to the current research by utilizing metalinguistic theory to identify more precise constructs that make up writing self-efficacy. At the same time the results of this study did not align themselves as closely to metalinguistic theory as was initially anticipated and it is important that we ascertain why. More research is needed to clarify these questions and strengthen future research results. Perhaps most important is a continued cross-disciplinary approach that utilizes linguistics and cognitive writing models as Alves (2012) suggests. Such an approach is most likely to produce progress in metalinguistics and translation during writing. 83
84
References
Alves, R. (2012). The future is bright for writing research. In V. Berninger (Ed.), Past,
present, and future contributions of cognitive writing research to cognitive
psychology, (pp. 591-598). New York: Psychology Press.
Anderson, J. R. (1996). ACT: A simple theory of complex cognition. American
Psychologist, 51, 355-365.
Anderson, J. R. (2000). Cognitive psychology and its implications. New York: Worth.
Arfé, B. (2012). Looking into the text generation box to find the psycholinguistic
(cognitive-language) writing process. In V. Berninger (Ed.), Past, present, and
future contributions of cognitive writing research to cognitive psychology, (pp.
573-578). New York: Psychology Press.
Almargot, D., & Chanquoy, D. (2001). Through the models of writing. Dordrecht, The
Netherlands: Kluwer Academic Publishers.
Almargot, D., & Chanquoy, D. (2012). Through the models of writing: Ten years after
and visions for the future. In V. Berninger (Ed.), Past, present, and future
contributions of cognitive writing research to cognitive psychology, (pp. 567-
572). New York: Psychology Press.
Almargot, D., & Fayol, M. (2009). Modeling the development of written composition. In
R. Beard, D., Myhill, J. Riley, & M. Nystrand (Eds.), The Sage handbook of
writing development (pp.23-47). London: Sage.
Austin, J. (1975). How to do things with words. Cambridge, MA: Harvard University
Press.
Baddeley, A. (1986). Working memory. Oxford, U.K.: Clarendon. 85
Baddeley, A. (2007). Working memory, thought, and action. New York: University of
Oxford Press.
Badia, M. (2000). Students’ conceptions on academic writing. In G. Rijlaarsdam & E.
Espéret (Series Eds.) & A. Camps & M. Milian (Vol. Eds.), Studies in Writing
Vol. 6. Metalinguistic activity in learning to write (pp. 49-78). Amsterdam:
Amsterdam University Press.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.
Englewood Cliffs, NJ: Prentice Hall.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman.
Bandura, A. (2006). Guide to constructing self-efficacy scales. In F. Pajares & T. Urdan
(Eds.), Self-efficacy beliefs of adolescents (pp. 307–337). Greenwich, CT:
Information Age.
Bangert-Drowns, R., Hurley, M., & Wilkenson, B. (2004). The effects of school-based
writing-to-learn interventions on academic achievement: A meta-analysis. Review
of Educational Research, 74(1), 29-58.
Bates, E. (1976). Language and context: The acquisition of pragmatics. New York:
Academic Press.
Becker, A. (2006) A review of writing model research based on cognitive processes. In
A. Horning and A. Becker (Eds.), Revision: History, theory, and practice. West
Lafayette, IN: Parlor Press
Beers, S. F., & Nagy, W. E. (2009). Syntactic complexity as a predictor of adolescent
writing quality: Which measures? Which genre? Reading and Writing, 22, 185–
200. 86
Benton, S., Corkill, A., Sharp, J., Downey, R., & Khramtsova, I. (1995). Knowledge,
interest, and narrative writing. Journal of Educational Psychology, Vol. 87(1),
(66-79).
Bereiter, C., & Scardamalia, M. (1987). The psychology of written composition.
Hillsdale, NJ: Lawrence Earbaum Associates.
Berninger, V. (1996). Reading and writing acquisition: A developmental
neuropsychological perspective. Boulder, CO: Westview Press.
Berninger, V., & Swanson, H. L. (1994). Modifying Hayes and Flower’s model of skilled
writing. In E. Butterfield (Ed.), Advances in cognition and educational practice,
Vol. 2, (pp. 57-81).
Berninger, V., Rijlaarsdam, G., & Fayol, M. (2012). Mapping research questions about
translation to methods, measures, and models. In M. Fayol, D. Alamargot, & V.
Berninger (Eds.), Translation of thought to written text while composing:
Advancing theory, knowledge, research methods, tools, and applications (pp.27-
70). New York: Psychology Press.
Blake, W. (1972). The tyger. In M. Abrams, G. Ford, & D Daiches (Eds.), The Norton
Anthology of English Literature Vol. 2 (p. 39). New York: Norton & Company.
Bong, M. (2006). Asking the right question: How confident are you that you could
successfully perform these tasks? In F. Pajares & T. Urdan (Eds.), Self-efficacy
beliefs of adolescents (pp. 287–303). Greenwich, CT: Information Age
Braddock, R., Lloyd-Jones, R., & Schoer, L. (1963). Research in written composition.
Urbana, IL: National Council of Teachers of English.
Bruning, R., Dempsey, M., Kauffman, D. F., McKim, C., & Zumbrunn, S. (2013). 87
Examining Dimensions of Self-Efficacy for Writing. Journal of Educational
Psychology. 105(1), (25-38).
Bruning, R., & Horn, C. (2000). Developing motivation to write. Educational
Psychologist, 35, 25–37.
Calkins, L., (1994). The art of teaching writing. Ontario: Heineman.
Campbell, D., & Fiske, D. (1959). Convergent and discriminant validation by the
multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
Camps, A, & Milian, M. (2000). Metalinguistic activity in learning to write: An
introduction. In G. Rijlaarsdam & E. Espéret (Series Eds.) & A. Camps & M.
Milian (Vol. Eds.), Studies in Writing vol. 6. Metalinguistic activity in learning to
write (pp. 1-28). Amsterdam: Amsterdam University Press.
Camps, A., Guasch, O., Milian, M., & Ribas, T. (2000). Metalinguistic activity: The link
between writing and learning to write. G. Rijlaarsdam & E. Espéret (Series Eds.)
& A. Camps & M. Milian (Vol. Eds.), Studies in Writing vol. 6. Metalinguistic
activity in learning to write (pp. 103-124). Amsterdam: Amsterdam University
Press.
Chenoweth, N. A., & Hayes, J. R. (2001). Fluency in writing: Generating text in L1 and
L2. Written Communication, 18, 80–98.
Creswell, J. (2002). Educational research: Planning, conducting, and evaluating
quantitative and qualitative research. Upper Saddle River, NJ: Pearson
Education. 88
Christie, F. (2010). The “Grammar Wars” in Australia. In T. Locke (Ed.), Beyond the
grammar wars: A resource for teachers and students on developing language
knowledge in the English/literacy classroom (pp. 55-72). New York: Routledge.
Croft, W., & Cruse, D.A. (2009). Cognitive linguistics. Cambridge: University of
Cambridge Press.
Crowhurst, M. (1990). Teaching and learning the writing of persuasive/argumentative
discourse. Canadian Journal of Education, 15(4), 348-359.
Cruse, A. (2004). Meaning in language: An introduction to semantics and pragmatics.
Oxford, England: Oxford University Press.
Dempsey, M., PytlikZillig, L., & Bruning, R. (2009). Helping preservice teachers learn to
assess writing: Practice and feedback in a Web-based environment. Assessing
Writing, 14(1), 38-61.
DeVellis, R. (2011) Scale development: Theory and applications. Los Angeles: Sage.
Elbow, P. (Ed.). (1994). Landmark essays on voice and writing. Mahwah, NJ: Lawrence
Erlbaum Associates.
Evans, V, & Green, M. (2006). Cognitive linguistics: An introduction. Mahwah, NJ:
Lawrence Erlbaum Associates.
Faigley, L. (1992). Fragments of Rationality: Postmodernity and the Subject of
Composition. Pittsburg, PA: University of Pittsburg Press.
Fallahi, C., Wood, R., & Austad, C. (2006). A program for improving undergraduate
psychology students’ basic writing skills. Basic Writing Skills, 33(3), 171-175.
Fauconnier, G. (1994). Mental spaces. Cambridge: Cambridge University Press. 89
Fayol, M. (2012). Cognitive processes of children and adults in translating thought into
write language in real time: Perspectives from 30 years of programmatic cognitive
psychology and linguistics research. In V. Berninger (Ed.), Past, present, and
future contributions of cognitive writing research to cognitive psychology, (pp.
27-60). New York: Psychology Press.
Fayol, M., Alamargot, D., & Berninger, V. (Eds.). (2012a). Translation of thought to
written text while composing: Advancing theory, knowledge, methods, and
applications. New York: Psychology Press.
Fayol, M., Alamargot, D., & Berninger, V. (Eds.). (2012b). Introduction to the book:
From cave writers to elite scribes to professional writers to universal writers,
translation is fundamental to writing. Translation of thought to written text while
composing: Advancing theory, knowledge, methods, and applications. New York:
Psychology Press.
Flower, L., & Hayes, J. (1980). The dynamic of composing: Making plans and juggling
constraints. In L. W. Gregg, & E. R. Steinberg (Eds.), Cognitive processes in
writing (pp. 31-50). Hillsdale, NJ: Lawrence Erlbaum Associates.
Flower, L., & Hayes, J. R. (1984). The representation of meaning in writing. Written
Communication, 1, 120 –160.
Geeraerts, D. (2010). Theories of lexical semantics. Oxford: Oxford University Press.
Gombert, J. E. (1992). Metalinguistic development. Chicago, IL: The University of
Chicago Press. 90
Graham, S. (2006). Writing. In P. Alexander & P. Winne (Eds.), Handbook of
Educational Psychology (pp. 457-478). Mahwah, NJ: Lawrence Earbaum
Associates.
Graham, S., Berninger, V., Abbott. R., Abbott, S., & Whitaker, D. (1997). The role of
mechanics in composing of elementary school students: A new methodological
approach. Journal of Educational Research, 92, 170-182.
Graham, S., & Harris, K. (2012). The role of strategies, knowledge, will, and skills in a
30-year program of writing research (with homage to Hayes, Fayol, and Boscolo).
In V. Berninger (Ed.), Past, present, and future contributions of cognitive writing
research to cognitive psychology, (pp. 177-196). New York: Psychology Press.
Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent
students. Journal of Educational Psychology, 99, 445–476.
Graesser, A., Hopkinson, P., Lewis, E., & Bruflodt, H. (1984). The impact of different
information sources on idea generation: Writing off the top of our heads. Written
Communication, 1(3), 341-364.
Grice, P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.), Syntax and
semantics Vol. 3: Speech acts. (pp. 41-58). New York: Academic Press.
Hacker, D., Dunlosky, J., & Graesser, A. (2009). A growing sense of “agency.” In D.
Hacker, J. Dunlosky, & A. Graesser (Eds.), Handbook of metacognition in
education (pp.1-4). New York: Routledge.
Hairston, M. (1982). The winds of change: Thomas Kuhn and the revolution in the
teaching of writing. College Composition and Communication, Vol. 33(1), 76-88. 91
Hall, C., Smith, P., & Wicaksono, R. (2011). Mapping applied linguistics: A guide for
students and practitioners. New York: Routledge.
Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. Essex, G.B.: Pearson.
Halliday, M. A. K., & Matthiessen, M. (2004). An introduction to functional grammar.
London: Hodder.
Harris, K., & Graham, S. (1992). Helping young writers master the craft: Strategy
instruction and self-regulation in the writing process. Cambridge, MA: Brookline
Books.
Hartley, J. (1991). Psychology: Writing and computers: a review of the research. Visible
Language, 25(4), 339-375.
Hartwell, P. (1985). Grammar, grammars, and the teaching of grammar. College English,
47(2), 105-127.
Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing.
The science of writing: Theories, methods, individual differences, and
applications. (pp. 1-28). Mahwah, NJ: Lawrence Erbaum Associates.
Hayes, J. R., & Cehnoweth, A. (2006). Is working memory involved in the transcribing
and editing of texts? Written Communication, 23(2), 135-149.
Hickmann, M. (1983). Reported speech: metapragmatic aspects of language and its
development. Nijmegen, The Netherlands: Max-Planck Institut für
Psycholinguistik, internally circulated document.
Hinton, P. (2004). Statistics explained. New York: Routledge.
Horn, J. (165). A rationale and test for the number of factors in factor analysis.
Psychometrica, 30, 179-185. 92
Jakobsen, R. (1960). Closing statement: Linguistics and poetics. T. Sebeok (Ed.), Style in
language. (pp. 350-377). New York: Wiley.
Karmiloff-Smith, A. (1981). A functional approach to child language. Cambridge:
Cambridge University Press.
Karmiloff-Smith, A. (1996). Beyond modularity: A developmental perspective on
cognitive science. Boston, MA: MIT Press.
Kellogg, R. (1987). Effects of topic knowledge on the allocation of processing time and
cognitive effort to writing processes. Memory & Cognition, 15, 256-266.
Kellogg, R. (1993). Observations on the psychology of thinking and writing.
Composition Studies, 21, 3-41.
Kellogg, R. (1994). The psychology of writing. New York: Oxford University Press
Kellogg, R. (1996). A model of working memory in writing. In C. M. Levy & S. Ransdell
(Eds.), The science of writing: Theories, methods, individual differences, and
application (pp. 57-71).
Kellogg, R. (2001). Long-term working memory in text production. Memory and
Cognition, 29(1), 43-52.
Kemper, S. (1987). Constraints on psychological processes in discourse production. In H.
W. Dechert & M. Raupach (Eds.), Psycholingistic models of production (pp. 185-
184). Norwood, NJ: Ablex Publishing Corporation.
Kintsch, W. (1987). Psychological processes in discourse production. In H. W. Dechert &
M. Raupach (Eds.), Psycholingistic models of production (pp. 163-180).
Norwood, NJ: Ablex Publishing Corporation. 93
Kintsch, W., & van Dijk T. A. (1978). Toward a model of text comprehension and
production. Psychological Review, 85(5), (363-394).
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago
Press.
Langacker, R. (1987). Foundations of cognitive grammar, Vol. 2. Stanford, CA: Stanford
University Press.
Langacker, R. (2008). Cognitive grammar: A basic introduction. Oxford: Oxford
University Press.
Lindemann, E. (2001). A rhetoric for writing teachers. New York: Oxford University
Press.
Matthews, P. (2010). Factors influencing self-efficacy judgments of university students in
foreign language tutoring. The Modern Language Journal, 94(4), 618-635.
McCarthy, P., Briner, S., Myers, J., Graesser, A., & McNamara, D. (2008). Are three
words all we need? Recognizing genre at the sub-sentential level. Proceedings of
the Cognitive Science Society (pp. 613-618). Washington D.C.: Cognitive Science
Society.
McCarthy, P., Meier, S., & Rinderer, R. (1985). Self-efficacy and writing: A different
view of self-evaluation. College Composition and Communication, 36, 465–471
McCutchen, D. (1986). Domain knowledge and linguistic knowledge in the development
of writing ability. Journal of Memory and Language, 25, 431-444.
McCutchen, D. (1996). A capacity theory of writing: Working Memory in composition.
Educational Psychology Review, 8(3), 299-325. 94
McCutchen, D. (2000). Knowledge processing and Working Memory: Implications for a
theory of writing. Educational Psychology, 23(1), 13-23.
McCutchen, D. (2011) From novice to expert: Implications of language skills and
writing-relevant knowledge for memory during the development of writing.
Journal of Writing Research, 3(1), (51-68).
McCutchen, D. (2012). Phonological, orthographic, and morphological word-level skills
supporting multiple levels of the writing process. In V. Berninger (Ed.), Past,
present, and future contributions of cognitive writing research to cognitive
psychology, (pp. 197-216). New York: Psychology Press.
McCutchen, D., Green, L., & Abbott, R. (2008). Children’s morphological knowledge:
Links to literacy. Reading Psychology, 29(4), 289-314.
McCutchen, D. Logan, B., & Biangardi-Orpe, U. (2009). Making meaning: Children’s
sensitivity to morphological information during word reading. Reading Research
Quarterly, 44(4), 360-376.
McCutchen, D., & Perfetti, C. (1983). Local coherence: Helping young writers manage a
complex task. The Elementary School Journal, 84(1), 70-75.
McNamara, D., & Grasser, A. (2011). Coh-Metrix: An automated tool for theoretical and
applied natural language processing. In McCarthy, P. & Boongthum-Denecke, C.
(Eds.), Applied natural language processing and content analysis: Identification,
investigation, and resolution (pp. 188-205). Hershey, PA: IGI Global.
Myhill, D. (2008). Towards a linguistic model of sentence development in writing.
Language and Education, 22(5), 271-288. 95
Myhill, D. (2009). Becoming a designer: Trajectories of linguistic development. In R.
Beard, D. Myhill, J. Riley & M. Nystrand (Eds.), The Sage handbook of writing
development. Los Angeles: Sage.
Myhill, D. (2010). Ways of knowing: Grammar as a tool for developing writing. In T.
Locke, (Ed.), Beyond the grammar wars: A resource for teachers and students on
developing language knowledge in the English/literacy classroom (pp. 129-148).
New York: Routledge.
Myhill, D. (2012). The ordeal of deliberate choice: Metalinguistic development in
secondary writers. In V. Berninger (Ed.), Past, present, and future contributions
of cognitive writing research to cognitive psychology, (pp. 247-274). New York:
Psychology Press
National Assessment of Educational Progress (2012). The nation’s report card: Writing
2007. Retrieved January 12, 2013, from
http://nationsreportcard.gov/writing_2007/
National Commission on Writing in America’s Schools and Colleges (2003). The
neglected “R”: The need for a writing revolution. Retrieved January 12, 2013,
from htttp://www.writingcommission.org/
National Writing Project (2003). Because writing matters: Improving student writing in
our schools. San Francisco: Jossey-Bass.
Nippold, M., Ward-Lonergan, J., & Fanning, J. (2005). Persuasive writing in children,
adolescents, and adults: A study of syntactic, semantic, and pragmatic
development. Language, Speech, and Hearing Services in Schools, 36, 125-138.
Nunally, J., Berstein, I. (1994). Psychometric theory. McGraw Hill. 96
Pajares, F. (2003). Self-efficacy beliefs, motivation, and achievement in writing: A
review of the literature. Reading and Writing Quarterly, 19, 139–158.
Pajares, F. (2007). Empirical properties of a scale to assess writing self- efficacy in
school contexts. Measurement and Evaluation in Counseling and Development,
39, 239–249.
Pajares, F., Britner, S. L., & Valiante, G. (2000). Relation between achievement goals
and self-beliefs of middle school students in writing and science. Contemporary
Educational Psychology, 25, 406–422.
Pajares, F., & Johnson, M. J. (1994). Confidence and competence in writing: The role of
writing self-efficacy, outcome expectancy, and apprehension. Research in the
Teaching of English, 28, 313–331.
Pajares, F., & Johnson, M. J. (1996). Self-efficacy beliefs in the writing of high school
students: A path analysis. Psychology in the Schools, 33, 163–175.
Pajares, F., Miller, M. D., & Johnson, M. J. (1999). Gender differences in writing self-
beliefs of elementary school students. Journal of Educational Psychology, 91, 50–
61.
Pajares, F., & Valiante, G. (1997). Influence of self-efficacy on elementary students’
writing. Journal of Educational Research, 90, 353–360.
Pajares, F., & Valiante, G. (1999). Grade level and gender differences in the writing self-
beliefs of middle school students. Contemporary Educational Psychology, 24,
390–405. 97
Pajares, F., & Valiante, G. (2001). Gender differences in writing motivation and
achievement of middle school students: A function of gender orientation?
Contemporary Educational Psychology, 26, 366–381
Pajares, F., & Valiante, G. (2006). Self-efficacy beliefs and motivation in writing
development. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook
of writing research (pp. 158–170). New York, NY: Guilford Press.
Patil, Vivek H., Surendra N. Singh, Sanjay Mishra, & D. Todd Donavan (2007). Parallel
Analysis Engine to Aid Determining Number of Factors to Retain [Computer
software]. Available from http://smishra.faculty.ku.edu/parallelengine.htm
Pedhazur, E. (1997). Multiple regression in behavioral research. Toronto: Thomson
Learning.
Pennebaker, J., & Booth, R. Francis, M. (2007). Linguistic Inquiry and Word Count
(version 2007), Austin, TX: University of Austin.
Rijlaarsdam, G., & Couzijn, M. (2000). Stimulating awareness of learning in the writing
curriculum. In G. Rijlaarsdam & E. Espéret (Series Eds.) & A. Camps & M.
Milian (Vol. Eds.), Studies in Writing Vol. 6. Metalinguistic activity in learning to
write (pp. 1-28). Amsterdam: Amsterdam University Press.
Rodriguez, M. (2008). Learning to appreciate English: An action-research: Reflections on
language valorization, self-efficacy, and metalinguistic awareness of basic ESL
students at the University of Puerto Rico. Dissertation Abstracts International,
68(11), pp. 45-98. 98
Roth, F. P., Speece, D. L., Cooper, D. H., & la Paz, S. (1996). Unresolved mysteries:
How do metalinguistic and narrative skills connect with early reading? Journal of
Special Education 30(3), 257-277.
Rubin, D., & Piché, G. (1979). Development in syntactic and strategic aspects of
audience adaptation skills in written persuasive communication. Research in the
Teaching of English, 13(4), 239-316.
Rumelhart, D., & McClelland, J. (Eds.) (1985). Parallel distributed processing vols. 1-2.
Cambridge, MA: MIT Press.
Searle, J. (1969). Speech acts: An essay in the philosophy of language. Cambridge:
University of Cambridge Press.
Searle, J. (1999). The future of philosophy. Retrieved January 13, 2013, from
w.scribd.com/doc/19935012/Searle-John-Future-of-Philosophy
Saddler, B., & Graham, S. (2007). The relationship between writing knowledge, and
writing performance among more or less skilled writers. Reading & Writing
Quarterly, 23(3), 231-247.
Schleppegrell, M. (2004). The language of schooling: A functional linguistics
perspective. Mahwah, NJ: Lawrence Erlbaum Associates.
Shell, D., Colvin, C., & Bruning, R. (1995). Developmental and ability differences in
self-efficacy, causal attribution, and outcome expectancy mechanisms in reading
and writing achievement. Journal of Educational Psychology, 87, 386–398.
Shell, D., Murphy, C., & Bruning, R. (1989). Self-efficacy and outcome expectancy
mechanisms in reading and writing achievement. Journal of Educational
Psychology, 81, 91–100. 99
Sheils, M. (1975, December 8). Why Johnny can’t write. Newsweek, 23, 58-63.
Swierzbin, B. (2010). Demonstratives’ special place in the English reference system:
Why that’s important for English language learners. Language and Linguistics
Compass, 4(10), 987-1000.
Thompson, G. (2004). Introducing functional grammar. London: Hodder.
Weaver, C. (1996). Teaching grammar in context. Portsmouth, NH: Heinemann.
Zimmerman, B. J., & Bandura, A. (1994). Impact of self-regulatory influences on writing
course attainment. American Educational Research Journal, 31, 845–862.
Zimmerman, B. J., & Kitsantas, A. (1999). Acquiring writing revision skill: Shifting from
process to outcome self-regulatory goals. Journal of Educational Psychology, 91,
241–250.
Zimmerman, B. J., & Kitsantas, A. (2002). Acquiring writing revision and self-regulatory
skill through observation and emulation. Journal of Educational Psychology, 94,
660–668.
Zwick, W. & Velicer, W. (1986). Factors influencing five rules for determining the
number of components to retain. Psychological Bulletin, 99, 432-442.
100
APPENDIX A
Demographic Survey
Please answer these questions to the best of your ability.
1. My estimated GPA is ______
2. I usually receive a grade of ______on my college papers
3. Gender: Male Female Age:______
4. Academic standing: Fr So Jr Sr.
5. What is your ethnicity? (Circle one)
African American
Asian/Pacific Islander
Caucasian
Latina/Latino
Native American
Other (e.g., multiracial)
Prefer not to answer 101
APPENDIX B
Metalinguistic Self-Efficacy Scale
Please read the tasks below and consider how successful you can be at them related to writing academic papers. Rate your confidence as a percentage using a scale of 0% to 100%. Zero percent equals “does not represent me at all,” 100% equals “represents me exactly. You can write ANY number in the 0-100% range that best describes you.
I can tell . . . Percent 1 . . . when to add prefixes and suffixes to words ______2 . . . when to use punctuation ______3 . . . if I understand my audience’s perspective on my topic ______4 . . . if I have provided good evidence for my argument ______5 . . . when to use singular and plural words ______6 . . . if my writing has the information my audience needs ______7 . . . when I should look for a better word ______8 . . . when my paragraph structures will impact my reader ______9 . . . which of two words better matches my meaning ______10 . . . if my words will impact my reader ______11 . . . when to blend multiple ideas into one sentence ______12 . . . if my grammar supports my writing ______13 . . . when my grammar is wrong ______14 . . . when to use spelling strategies I have learned ______15 . . . when I have formed a strong argument ______16 . . . when to use ‘this’ and ‘that’ to refer to other words ______17 . . . when to choose different sentence structures ______18 . . . when my argument needs to be clearer ______19 . . . if I have used the best argument structure ______20 . . . if I am making typing errors ______21 . . . when to apply spelling rules ______22 . . . if my pronouns refer clearly to my nouns ______102
23 . . . when to break words into syllables ______24 . . . if I have corrected all my spelling ______25 . . . when my words express my ideas exactly ______26 . . . if my written argument reaches my writing goals ______27 . . . when to use new words for a previous word ______28 . . . when to use short and long sentences for effect ______29 . . . when to apply grammar rules ______30 . . . if my writing is interesting and engaging ______31 . . . when to incorporate new ideas into my writing ______32 . . . if I have included enough ideas in my writing ______33 . . . when I should add more descriptive words ______34 . . . if my writing is convincing ______35 . . . if my audience will follow my organization ______
103
APPENDIX C
Reading Assignment for Essay
Washington Times, Seattle WA, January, 28 2013
BACKGROUND Seattle high school teachers are boycotting a district mandated standardized examination, the Measures of Academic Progress, or MAP, which is presumed to measure math, reading, and other academic skills. The protest is gaining national support from teachers. The Chicago Teachers Union, for example, announced their support to the boycott on January 28, 2013.
Resentment toward standardized tests increased when it became the cornerstone of the No Child Left Behind act under the Bush Administration. More recently, President Obama has supported standardized tests and supported tying them to teacher evaluations. Secretary of Education Arne Duncan, however, has criticized overreliance on test preparation, a major complaint of teachers.
TEACHERS’ POSITION Seattle teachers are frustrated that the MAP doesn’t line up with the curriculum, doesn’t provide meaningful feedback teachers can use to improve learning, and ties up computer labs and classrooms for test-taking.
Teachers and other critics say the MAP is a waste of time and unfair to low-income and minority students.
Seattle teachers are defying a threat of a 10-day unpaid suspension.
CRITICS’ POSITION Critics of the Seattle teachers claim the teachers’ main fear is that students’ performance on the MAP will reflect badly on them.
Michael Petrilli, executive vice president of the conservative Thomas B. Fordham Institute, had this to say: “Ostensibly [the Seattle teachers] protest is about the overuse of tests, the instructional time that those tests devour, and the culture of soulless data-driven instruction that animates today’s brand of school reform. The real reason the [teachers] attack the MAP one must presume, is because it’s a small part of Seattle’s new teacher- evaluation system. . . . It can pinpoint precisely where students are on the achievement spectrum and can give teachers full credit for any progress.”
104
APPENDIX D
Essay Prompt Used to Generate Writing Sample
Write a letter to Superintendent Jose Banda explaining your position, opinions, and feelings on this boycott and the issues that underlie it.
You must convince the Superintendent of your position. He will, like most adept readers, be affected by the following writing related abilities: • Spelling • Sentence structure and fluidity • Words and ideas • Attention to audience and appropriateness of evidence • How interesting and tied together your sentences and your text overall are
105
APPENDIX E
Operationalized Rubric for Scoring Participants’ Essays
Metaphonology • Spelling/Capitalization (error count) • Syllabilzation correct • Fluency (Number of typing errors. For example, type word right sometimes, and wrong others.) • Figurative writing (e.g., cadence , rhyming: alliteration, consonance, assonance, syllabic, oblique, slant rhyme)
Metasyntax • Punctuation (error count) • Syntactic (grammatical) accuracy (error count) • Variation/manipulation of syntax. That is, is there evidence of intention to affect the audience. Does affect on your reaction to the text? • Syntactic density (t-units—a dominant clause and all its subclauses)
Metasemantics • Number of sophisticated words/ Number of academic words (Word—reading level; check mark words 3 or more syllables) • Figurative writing (e.g., analogy, simile, exemplification, compare and contrast, metaphor, irony, hyperbole, etc.)
Metapragmatics • Meets audience need for relevant information • Writing is appropriate for rhetorical context (fits or compliments genre and audience) • Anticipates audience questions, concerns, counterarguments • Evidence is appropriate and convincing. Ideational warranting (number of ideas and presence of supporting ideas) • Author achieves stated goals, affects readers’ thinking
Metatextual • Text is coherent overall • Text is cohesive (i.e., transitions at local level; accurate use of “substitute” words) • Variation in sentence length • Variation in paragraph length • Evidence of a rhetorical schema
106
APPENDIX F
Holistic Rubric for Scoring Participants’ Essays
Five Points • No spelling or capitalization errors • No punctuation and syntax (grammar) errors • Sentences frequently contain high number of t-units • Word shows college reading level and number of words • Shows clever use of sound, words, and sentence structure • Writing addresses topic with pertinent and convincing evidence • Writing meets audience needs and expectations • Writer meets stated goals • Writing makes good sense • Writing is easy to follow because of use of transitions • Sentences and paragraphs vary in length and structure • Writer uses one or more rhetorical devices Three points • Occasional spelling or capitalization errors • Occasional punctuation and syntax (grammar) errors • Sentences occasionally contain multiple t-units • Word shows middle school reading level • Shows some utilization of sound, words, and sentence structure for effect • Writing on average addresses topic with evidence • Writing on average meets audience needs and expectations • Writer on average meets stated goals • Writing does not sway the reader one way or other • Writing generally uses transitions • Sentences and paragraphs show some variance in length and structure • Writer does not use rhetorical devices effectively One point • Frequent spelling or capitalization errors • Frequent punctuation and syntax (grammar) errors • Sentences are short with few t-units • Word shows low grade-reading level and low number of words overall • Writing does not show any use of sound, words, and sentence structure for effect • Writing does not address topic with pertinent and convincing evidence • Writing does not meet audience needs and expectations • Writer does not meet stated goals • Writing does not make sense • Writing does not use transitions • Sentences and paragraphs do not vary in length and structure • Writer does not use any rhetorical devices 107
APPENDIX G
Recruiting Script
My name is Mike Dempsey, and I would like to offer you an opportunity to participate in a writing research study. The study will be scheduled for a time and place outside your class time, and the entire study will not take more than 1 hour to complete.
During the study, you will perform three activities: • First, you will be asked to read a real news story about an event involving teachers and testing. • Second, you will be asked to write a letter to the Superintendent of the school district where this story takes place. • Third, you will be asked to report your beliefs about specific writing tasks. There are 35 items you will respond to.
The purpose of this research is to better understand how writers’ beliefs about their writing relate to their writing. You may find participation in this study interesting and enjoyable, and the activities may prompt you to think more deeply about your own writing. Furthermore, you will have an opportunity to learn more about educational research.
You are not required to participate in this study, and you have the right at any time to withdraw from the study. Non-participation or withdrawal will not adversely affect your grade in this course, your relationship with your instructor, the researcher, or the University of Nebraska.
I will now hand out a list of dates and times you may complete the study.
Are there any questions?