Online Reading Comprehension 1

Running head: ONLINE READING COMPREHENSION AND EXECUTIVE FUNCTION

Online Reading Comprehension and Executive Function: What is the Relationship?

[EPET student]

Michigan State University Online Reading Comprehension 2

Abstract

The proposed correlational study will investigate the relationship between a measure of online reading comprehension and measures of executive function (EF) in a sample of 50 typically developing 8th graders while holding constant measures of offline reading comprehension and time spent online. The dependent variable, online reading comprehension, will be measured using the ORCA-Iditarod (revised) task (Leu & Reinking, 2005). Students’ 8th grade MEAP reading comprehension scores will be taken as indicators of offline reading comprehension ability. Time spent online will be measured using an adapted time-diary method based on the work of Nie and Hillygus (2002). To address the “task impurity problem” that arises with psychometric tests that also measure cognitive functioning beyond the variables of interest, three foundational EFs: response inhibition, updating working memory and set-shifting, will be measured with two instruments each. This will allow for common variance between the tasks to be identified and used in the statistical model. Because of the small sample size and the exploratory nature of the proposed study, partial least squares structural equation modeling will be used to construct latent variables and to characterize the relationships that exist between the dependent and independent variables. Theory suggests (Duke, Schmar-Dobler & Zhang, 2006) that online reading comprehension may make additional cognitive demands of the reader but there is little empirical evidence to support this perspective or that quantifies the nature of those demands. This study aims to address this need. Moreover, for adolescents, it is particularly important to understand if executive function plays a role in online reading comprehension since

EF continues to develop through to adulthood. If significant, findings from this study could potentially inform the development of new literacies curricula for teens. Online Reading Comprehension 3

Introduction

If you read the popular press, you’ve probably read the doomsday accounts. You’ve probably read that the Internet is making kids dumber (Bauerlein, 2008b) and that because of the

Internet, we’re collectively losing our ability to read critically (Bauerlein, 2008a; Carr, 2008).

And yet, as Motoko Rich noted in the New York Times, (Rich, 2008) engagement with the web in any form means that readers must engage with text. There is a certain irony here. How can continued engagement with text turn people into less capable readers when even popular accounts of expertise (e.g., Gladwell, 2008) concede that experience forges it?

The contradiction may, in fact, come down to a definition. In an age of rapid technological change, how do – or perhaps even how should -- we define reading, anyway? Is reading the newspaper that arrives at your doorstep the same as reading the newspaper online? Is reading a textbook, a travel guide, a novel, or a short story in print the same as reading websites or even digital forms of the same text? These questions have been hotly debated in the field of literacy research (cite) and yet, there is general consensus on the following point: whether on the

Internet or in a printed book, if the reading is for information, for a specific purpose, or for learning, the reading process is necessarily strategic (Coiro, 2007; Zhang & Duke, 2008) and effortful (Duke, Schmar-Dobler, & Zhang, 2006; Pressley & Afflerbach, 1995; RAND Reading

Study Group, 2002). People skim the Internet for different purposes too (Zhang & Duke, 2008) and it may well be that the Internet as a medium facilitates or even demands a “skimming” habit.

But here’s the critical point. If 21st century teens are going to truly benefit from the power of the

Internet, they can’t stop at skimming. As their primary source of information (Ito, et al., 2008) teens must learn how to read the Internet critically (Donald J. Leu, 2000; Donald J. Leu, Kinzer,

Coiro, & Comack, 2004). The trouble is--and perhaps this is what the popular press has tapped Online Reading Comprehension 4 into--reading critically on the Internet is very difficult.

Emerging research in the field of online reading comprehension suggests that reading for information on the Internet may, in fact, be more complex than its traditional print-based analogue (Coiro, 2008; Coiro & Dobler, 2007; Hartman, Morsink, & Zheng, in press; Roswell &

Burke, 2009) and by extension, Internet reading may place additional demands on cognitive processes. In particular, mounting evidence suggests that online and print-based reading (herein referred to as offline reading) make differential demands on executive function (EF) (Coiro,

2007, 2008; Johnson, 2008; Small, Moody, Siddarth, & Bookheimer, 2009), which can be broadly defined as a cluster of higher-order cognitive skills that includes inhibition, set switching, updating working memory, planning, organizing and self-monitoring (Cutting,

Materek, Cole, Levine, & Mahone, 2009; Friedman, et al., 2008; Sesma, Mahone, Levine, Eason,

& Cutting, 2009; Strauss, Sherman, & Spreen, 2006). For teens, whose brains undergo significant re-organization, particularly in the pre-frontal cortex where EF is thought to be coordinated, (Geidd, 2004; Hill & Schneider, 2006; Kuhn, 2006; Parris, 2008) the complex demands of Internet reading experiences over time could make a significant impact on their cognitive development.

It is generally accepted that EF and, more specifically, its three most-researched subcomponents, (1) inhibition, (2) working memory and (3) task switching, are significant correlates of academic achievement in general (Best, Miller, & Jones, 2009) and offline reading comprehension in particular (Cutting, et al., 2009; Protopapas, Archonti, & Skaloumbakas, 2007;

Sesma, et al., 2009). Tangentially, work on the teaching of online reading comprehension strategies to middle-school students (e.g., Donald J. Leu, et al., 2008) seems to include the implicit assumption that EF plays a significant role in the process and yet, the teaching of Online Reading Comprehension 5 metacognitive strategies for online reading comprehension could potentially benefit from a more targeted understanding of the specific underlying cognitive mechanisms that mediate the online reading comprehension process. If individual differences in online reading comprehension are attributable, in part, to individual differences in EF, then teaching strategies that target the development of EF may have significant pay-off for learners (see Cartwright, 2002 for an example of teaching flexibility/task-switching for reading.) The first step, however, is to gain some initial insight into the relationships that may or may not exist between measures of online reading comprehension and measures of EF. Observational research has identified strategies and behaviors that seem characteristic of online readers with different levels of proficiency (Coiro,

2003; Coiro & Dobler, 2007; Wilber, 2008; Wyatt-Smith & Elkins, 2008; Zhang & Duke, 2008) but to date little work has been done to understand the underlying executive functions that may be mediating the strategic processes of online readers (Johnson, 2008)

The proposed correlational study is therefore designed to provide a preliminary examination of the relationships between measures of EF and online reading comprehension in a sample of young adolescents, a population for whom online reading comprehension is a critical skill but for whom the literature on EF is limited both generally (Best, et al., 2009) and within the domain of online reading comprehension.

Theoretical Framework

Though the proposed study could be framed by several theoretical frameworks, I have chosen to orient the study and its design within the following three perspectives: (1) Reading as a complex metacognitive process, (2) Dual-coding cognitive perspectives of strategy development and executive function (Kuhn & Pease, 2010) and (3) New literacies (Donald J.

Leu, et al., 2004). Summaries of each theory will be presented in sequence. Online Reading Comprehension 6

Reading Comprehension as a Complex Metacognitive Process embedded in Socio-Cultural

Context

Consistent with other recent studies of online reading comprehension (Coiro, & Dobler,

2007; Coiro, 2008) the proposed study also positions itself within the theoretical view of reading as a complex, multi-faceted process of making meaning (Pressley & Afflerbach, 1995; RAND

Reading Study Group, 2002). This theoretical view contends that readers--both online and offline--actively and purposefully engage with text, drawing from a complex array of metacognitive strategies and prior knowledge to construct meaning and that all of these cognitive processes are is embedded in and therefore influenced by a socio-cultural context from which the reader also derives meaning.

Dual Processing Model of Learning and Strategy Use

Through microgenetic analyses of strategy development over time, work by Siegler

(Siegler, 1995, 2007; Siegler & Chen, 1998) and Kuhn (Kuhn & Dean, 2005; Kuhn & Pease,

2010) has furnished the largely accepted view that over time, learners gradually become more sophisticated in their use of increasingly advanced strategies. Cognitive development from novice to expert is never linear, however. Siegler (1996) has characterized the process of strategy development as a series of overlapping waves. He suggests that children develop more advanced strategies with continued experience and engagement but that the process of “advancement” also involves regression to previously used, less-advanced approaches as they learn. Learning, then can be viewed as a back-and-forth process. This variability within learners and among different learners is attributable to the differential selection of strategies over time (Siegler, 2007). Kuhn and Pease (2010, p. 135) note that, “The period of time in which a mixture of more and less advanced strategies are applied variably may be prolonged.” Online Reading Comprehension 7

Though work by Siegler and Kuhn has focused largely on the acquisition of numeracy

(Siegler, 1995; Siegler, Thompson, & Opfer, 2009) and thinking for scientific inquiry (Kuhn,

1989; Kuhn & Dean, 2005) reading, itself, (as noted above) has long been characterized a strategic process (Baker & Brown, 1984; Israel, 2008; Pressley & Afflerbach, 1995; RAND

Reading Study Group, 2002; Stromso, Braten, & Samuelstuen, 2003). Recently, Sharp, Sinatra and Reynolds (2008) have applied the theoretical framework of overlapping waves to characterize children’s spelling development, suggesting that this cognitive view may, in fact, explain development of the strategic processes critical for literacy as well.

Building from the foundations of variability and overlapping waves in strategy development, Kuhn and Pease (2010) propose that strategy development is more complex than just “acquiring expertise” (p. 156). They suggest the involvement of a “metalevel manager” (p.

154) that coordinates both the production and control of appropriate strategies and the inhibition of less sophisticated strategies to which the learner has access. In the proposed study, this meta- level manager is considered synonymous with executive function.

This model is particularly salient because it is grounded in developmental variability. It acknowledges that there will be differences among learners and within learners as they become more strategic and it explains why the variability exists. Further, it explains how the variablility, over time, is coordinated by the “meta-level manager” into increasingly resilient and sophisticated strategies. This theoretical lens also grounds the practical implications of the study.

Kuhn and Pease have demonstrated the role that teaching and “social context” (p. 139) play in the progressive ability to coordinate appropriate and inappropriate problem-solving strategies.

Though an extrapolation to the case of online reading comprehension, their dual-coding theory suggests that teaching may support executive function and by extension, the observable Online Reading Comprehension 8 coordination of strategic behavior.

New Literacies Framework

The New Literacies Framework (Donald J. Leu, et al., 2004) asserts that new technologies require new literacy skills. Leu and colleagues (Donald J. Leu, et al., 2008; Donald

J. Leu, et al., 2007) suggest that online reading comprehension, in particular, requires readers to simultaneously access five new literacies skills: generating questions, locating information, evaluating information, synthesizing information and communicating information. Most importantly, however, the ability to carry out these skills must be grounded in an understanding of the deictic nature of technology. Coiro, Knobel, Lankshear and Leu (2008) suggest that the very definition of literacy acquisition is necessarily shifting because the texts and the technologies within which they’re embedded are constantly shifting too. They note that literacy acquisition can no longer be defined as “acquiring the ability to take advantage of the literacy potential inherent in any single, static, technology of literacy (e.g., traditional print technology) but rather by a larger mindset and the ability to continuously adapt to the new literacies required by the new technologies that rapidly and continuously spread on the Internet.” (p. 5).

The New Literacies Framework has been specifically conceptualized to focus on the literacy skills and mindsets required of students and citizens in the complex, digital literacy landscape of the 21st century. Underlying this framework is the view that these skills can and should be taught in schools (Leu, et al., 2008). For this reason, it has been selected as the most appropriate online reading framework for the currently proposed study because in the long term, the practical significance of the proposed study includes its application to the development of new literacies curricula for middle- and high-school students. Online Reading Comprehension 9

Research Question

Given that an understanding of the relationships between executive function and students’ ability to make meaning from what they read on the Internet could inform the development of more targeted new literacies curricula for adolescents, the proposed study asks the following question:

For a diverse group of typically developing 8th graders, what is the relationship between online reading comprehension ability and executive function while holding constant time spent online for different purposes and offline reading comprehension ability?

Online reading comprehension ability will be measured by the Online Reading

Comprehension Assessment (ORCA)-Iditarod (Revised) (Leu & Reinking, 2005). Executive function will be divided into three subcomponents. Response inhibition will be measured by the

Stroop Color-Word Test (Stroop, 1935) and a stop-signal test (Friedman, et al., 2008). Updating working memory will be measured by the CANTAB Spatial Working Memory Test (cite) and a keep-track test (Friedman, et al., 2008). Set shifting will be measured by the Wisconsin Card

Sorting Test (Berg, 1948; Grant & Berg, 1948) and a category switch test (Friedman, et al.,

2008). Time spent online will be measured by averaging two time diary accounts modeled after a method used by Nie and Hillygus (Nie & Hillygus, 2002). Offline reading comprehension will be measured by students’ 8th grade reading scores on the Michigan Education Assessment

Program (MEAP) test. Table 1, presented in the Method section, summarizes the proposed dependent and independent variables and the instruments that will be used to collect dependent and independent measures.

Review of Literature Online Reading Comprehension 10

The review of literature will begin with a brief overview of online reading comprehension, the dependent variable, and continue with a discussion of the five independent variables in the proposed statistical model: (1) offline reading comprehension, (2) time spent online, (3) response inhibition, (4) updating working memory representations, and (5) set shifting. Note that variables 3, 4 and 5 are considered subcomponents of the overarching construct of executive function (EF) (Best, et al., 2009; Friedman, et al., 2008).

Online reading comprehension

Duke, Schmar-Dobler and Zhang’s (2006) model of online reading comprehension looks like a Venn diagram (p. 318). One oval represents comprehension processes for printed texts, the other oval represents comprehension processes for electronic texts (including hypertexts) and the intersection of the two ovals represents shared reading processes. The authors note (p. 319) that skilled hypertext readers employ many of the same strategies as skilled offline readers (e.g., setting reading purpose, activating prior knowledge, predicting, previewing, attending to text structure, evaluating text and monitoring) but emphasize that unlike the comprehension of printed text that follows a generally linear process and is constrained by the finite nature of the paper-based medium, “hypertext comprehension demands the reader take an active role in text construction by forcing decisions about what to click […]” (p. 320). Further Duke et al. suggest that the complex array of choices on the open web for readers may cause “cognitive overload”

(p. 320) and require more “mental energy”. This is an important hypothesis.

Coiro, too, has suggested that online reading comprehension requires an extended set of cognitive skills (Coiro, 2003). In her dissertation, Coiro (2007) conducted a comparative case- study of three online readers with different levels of online reading comprehension ability--low, medium and high--and found that readers needed not only additional web-specific skills to make Online Reading Comprehension 11 meaning from online text, but that offline and online reading abilities can develop along seemingly independent trajectories. During online reading comprehension tasks, the most capable online reader in Coiro’s study strategically focused her attention to important contextual cues online, stayed focused on important aspects of the task demands, focused on content rather than process, and monitored and adjusted her understanding of what she read. During the same online reading tasks, however, the weakest online reader, was confused about where to focus his attention, often repeated use of a single ineffective strategy based on partially relevant or irrelevant task cues and was often distracted by personally interesting information that was not closely related to the task. Interestingly, a third student was found to be a very competent online reader but her offline reading comprehension skills were poor. The findings in this qualitative part of Coiro’s study raise several critical questions about the unique skill set required of online readers and the role that EF plays in students’ developing ability to coordinate those skills. Her qualitative analysis of the mid-level online reader suggests that offline and online reading skills may not always develop together which lends support to the theoretical view (Duke, et al., 2006) that online and offline reading comprehension skills are different in important ways. Further, the observed variability in strategy use also appears consistent with the developmental dual-coding model proposed by Kuhn and Pease (2010) which emphasizes the role of EF in coordinating the production and inhibition of appropriate strategies at the right time.

Offline reading comprehension vs. Online reading comprehension

It is important to make two further points of clarification regarding the proposed conceptualization of online reading comprehension, particularly as it relates to offline reading.

First, though Coiro’s case study highlighted a certain separability of online and offline reading comprehension skills (2007), it is assumed that offline reading is a foundational constituent of Online Reading Comprehension 12 online reading. It is assumed that students have acquired fundamental reading skills such as decoding, syntactic and semantic knowledge and fluency through their lifelong engagement with different kinds of print-based texts and that these faculties permit, to some extent, the reading of online text. Coiro’s study provides some empirical support for this assumption (2007). She found that offline reading comprehension correlated with scores on two versions of the ORCA test

(ORCA-I, r(109) = .612, p<.001 and ORCA-II, r(109) =.592, p<.001). Using hierarchical regression analysis, she found that offline reading comprehension explained 35.1 percent of the variance in online reading comprehension, which was significant (F-change (1, 107) = 57.812, p <.001)

(Coiro, 2008, p. 11). She concluded that offline reading comprehension contributed a significant amount of variance to her model of online reading comprehension. For this reason, a measure of offline reading comprehension has been included in the currently proposed model of online reading comprehension as well.

Second, it should be explicitly stated that the proposed definition of online reading is not simply the decoding of text on a computer screen, nor is it reading of text for that uniquely aesthetic experience of curling up with a good book in bed. In this case, the definition of online reading refers to the ability to question, locate, evaluate, synthesize and communicate extracted meaning from multi-modal, hyperlinked, informational text on the Internet (Donald J. Leu, et al.,

2004). Further, and like reading in any context (RAND Reading Study Group, 2002), the online reading process is assumed to be directed by a specific learning goal or task (Zhang & Duke,

2008). This conceptualization of online reading could be considered analogous to reading for information by picking some books off the library shelf, thinking about the meaning of what you read, how what you’ve read fits together and how this information helps you accomplish your learning goal – a process that has been studied in depth (e.g., Many, Fyfe, Lewis, & Mitchell, Online Reading Comprehension 13

1996; Stromso, et al., 2003). Online reading, then, is not just decoding letters and simple meaning. It is very much about the processes of finding and constructing meaning from what has been read. In fact, Coiro (2009) characterizes it as a “complex online problem solving process”

(p. 1).

Time spent online for different purposes

The inclusion of this variable is intended to account for differences in online reading comprehension due to practice effects and, potentially, a kind of expertise that Endsley (2006) calls situation awareness. Defined as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future” (Endsley, 2006, p. 634), situation awareness is domain specific and develops with protracted experience in that domain over time. Just like Wayne Gretzky who could famously understand the patterns of hockey and position himself not where the puck had been but rather, where the puck was going to be, more experienced Internet users may have developed a situation awareness that could reduce the cognitive load of the online reading comprehension task and free up resources for the hard thinking demanded by the ORCA-Iditarod task.

How much time do teens spend online? According to a Kaiser Family Foundation report, teens aged 11-14 spend an average of one hour and two minutes per day on computers for various purposes (Rideout, Roberts, & Foehr, 2005, p. 30). Feltovitch, Prietula and Ericsson

(2006, p. 60) note that 50 hours of engagement in a given activity is usually enough time to learn the “underlying structure of the activity” and the “aspects they must attend to” so that performance is acceptable and automated enough so as to require only minimal cognitive investment. Expertise, on the other hand is a “long-term developmental process” (p. 46) that emerges from rich, diverse and protracted experiences in a domain over an extended period of Online Reading Comprehension 14 time – perhaps as long as 10 years (Feltovich, et al., 2006, p. 60). For teens as a general population, it is certainly possible that by 13 or 14 they have accumulated nearly enough hours on computers to make them “experts” and with an average of one hour/day on computers, most teens probably have reached the 50-hour threshold for satisfactory performance in this domain.

Since time is a correlate of expertise then, the inclusion of the time variable in the proposed model is an attempt to control for differences in online reading comprehension performance resulting from time spent reading online for different purposes.

Executive Function

Miyake et al. (2000) have proposed that EF is both a unified and diversified construct (p.

87). This view suggests that EF consists of related, but separable components and that a common mechanism (or mechanisms) supports all EF processes (Best, et al., 2009, p. 183). This framework is supported by (a) developmental research that has shown different EF subcomponents to be important at different times across the lifespan (e.g., Zelzao, Craik, &

Booth, 2004) and for different cognitive profiles (e.g., deFrias, Dixon, & Strauss, 2009), (b) neuroimaging research that shows differential engagement of different areas of the pre-frontal cortex on different kinds of EF tasks (e.g., Gruber & Goschke, 2004), (c) clinical studies that have documented success on different EF tasks for patients who have sustained frontal-lobe injury (e.g., Duncan, Johnson, Swales, & Freer, 1997) and (d) studies of specific learning disabilities that seem to target some but not all EFs. For instance, Nigg, Carr, Martel and

Henderson (2007) have found several types of inhibitory control but that Attention Defecit

Hyperactivity Disorder (ADHD), for instance, is related to only two distinct types of control processes –effortful control related to innatention-disorganization and reactive control, related to hyperactivity-impulsivity (p. 260). ADHD is not, however, associated with deficits in perceptual Online Reading Comprehension 15 inhibition, and only partially related to response inhibition.

Though inhibition, updating working memory and set shifting are not the only subcomponents of EF, Miyake et al. (2000, p. 55) note that all three constructs have been

“postulated in the literature” as essential EFs. Further, they characterize them as “relatively circumscribed” and easy to define operationally (in comparison to more complex EFs such as planning, for instance), and suggest that several “conventional” tests of EF tap these mechanisms. Finally, they suggest that a good understanding of these three EFs is essential because (a) they are probably implicated in the performance of higher-order EFs (such as planning) and (b) they provide a basis for specifying the construct validity of conventional EF tests. This constrained, tri-partite operational definition of EF seems to have resonated with the larger field of research. Several other studies have maintained this basic definitional premise

(e.g., Best, et al., 2009; Friedman & Miyake, 2004; Friedman, et al., 2008) and for this reason, I too, have adopted this framework for defining the EF constructs in the proposed study. A review, including definitions, of each of these three foundational EFs follows.

Response inhibition. In an attempt to provide an overarching definition of cognitive inhibition that can be broadly applied to a range of mental processes, MacLeod (2007) defines cognitive inhibition as “the stopping or overriding of a mental process, in whole or in part, with or without intention.” (p.5). He acknowledges, however, that there are different kinds of inhibition. Nigg (2000) delineated four “executive inhibition” mechanisms (p. 237). He defines interference control as the ability to prevent interference due to resource or stimulus competition and suggests that this kind of interference is measured by the Stroop Test (Stroop, 1935). He defines behavioral inhibition as the ability to suppress pre-potent responses and suggests that the stop-signal task is a measure of this construct. Friedman and Miyake’s (2004) latent variable Online Reading Comprehension 16 analysis of pre-potent response inhibition (i.e. behavioral inhibition) and resistance to distractor interference (i.e. interference control), however, determined this theoretical separation to be an overextension. They collapsed these two variables into one latent variable called Resistance to pre-potent interference but found a third variable, resistance to proactive interference, to be a separate construct. So, while Nigg’s (2000) taxonomy may not have stood up to Friedman and

Miyake’s (2004) statistical analysis, their study still makes the same point as Nigg’s – that inhibitory processes are perhaps unified but diverse. In subsequent work, Friedman and Miyake

(Friedman, et al., 2008, p. 201) have defined response inhibition as the ability to inhibit dominant, automatic, or prepotent responses – and it is this operational defintion to which the currently proposed study ascribes.

Beyond the theoretical argument that inhibitory processes are fundamental to EF (Best, et al., 2009) it is important to consider how inhibitory processes may be involved in online reading comprehension. Given the infinite potential for distraction on the Internet, it would seem that the ability to suppress the pre-potent tendencies to read distractor items on a webpage or to follow links of interest that may not actually be related to the task at hand would predict, on some level, proficiency in online reading comprehension. Moreover, for tasks that require synthesis of pertinent information collected across a wide range of online sources, it would seem important to suppress irrelevant information and hold only important ideas in mind through the processes of search, evaluation and synthesis. Developmental research has revealed that the ability to inhibit develops through childhood and adolescence (Best, et al., 2009; Davidson, Amso, Anderson, &

Diamond, 2006; Huizinga, Dolan, & Molen, 2006). Though performance on stop-signal tasks improves through childhood and seems to level off by adolescence (Huizinga, et al., 2006), improvement on the Stroop test continues through to early adulthood (Best, et al., 2009, p. 186). Online Reading Comprehension 17

These findings lend support to the unity but diversity hypothesis of EF (Miyake, et al., 2000) but also suggest that teens may benefit from teaching that supports the development of inhibitory control through adolescence. Even if the effects of the teaching are not seen immediately, explicit support for a mechanism that is changing at this developmental change may benefit teens in the long run. For these reasons, it is important to consider the relationship between a student’s inhibition and online reading. If inhibition is found to play a significant role in the online reading comprehension process, research might then turn to questions of how to support its development most effectively.

Updating working memory. Baddeley’s multicomponent model of working memory

(WM) includes four components, (1) an attentional controller, the central executive, and three temporary storage systems, (2) the visuospatial sketchpad, (3) the phonological loop, and (4) the episodic buffer which is a more general, integrated storage system (Baddeley, 2007, p. 13). Any sort of problem solving process necessarily involves memory, and Baddeley’s working memory model offers a cognitive account for the holding of information in mind while manipulating, changing and updating it to accomplish a goal. Like inhibition and set-swtiching, updating working memory, defined as the ability to hold information and update it based on new information, is considered an essential EF (Best, et al., 2009; Friedman, et al., 2008; Miyake, et al., 2000). Also like other EFs, the ability to hold information in working memory and update it improves with age. Work by Gathercole and colleagues (Gathercole, Pickering, Ambridge, &

Wearing, 2004; Jarvis & Gathercole, 2003) has shown that performance on WM tests improves from age 4 to 15 and that better WM performance is related to better performance on tests of academic skills. Zelzao et al. (2004, p. 169) hypothesize that development in EF progresses as children become more able to manipulate increasingly complex and hierarchical rules for solving Online Reading Comprehension 18 problems, a process embedded in the development of working memory. Moreover, neuroimaging studies have discerned significant differences in brain activity between children, adolescents and adults on working memory tasks, suggesting a progressive refinement in the underlying neurological substrate for WM (Scherf, Sweeney, & Luna, 2006). And perhaps most importantly for the current study, better working memory capacity has been found to correlate with better reading comprehension in children aged 7-11 (Cain, Oakhill, & Bryant, 2004), presumably because better WM permits the integration of a more complex range of knowledge and skill.

Translated to the context of reading on the Internet, and particularly if it is defined as a complex problem solving process, it is therefore hypothesized that measures of WM will also contribute significant and unique variance to the measure of online reading comprehension.

Set Shifting. The third EF of interest, set shifting, is defined as “the ability to shift between mental states, operations, or tasks” (Best, et al., 2009, p. 187). Monsell (2003) calls it

“task switching” but effectively, the terms mean the same thing. To illustrate the process,

Monsell describes a professor sitting at his computer, trying to write a paper. The phone rings, he answers. It’s an administrator asking for a completed form. The professor looks around, finds it, takes the form to the administrator’s office, exchanging pleasantries with a colleague whom he passes along the way (p. 134). As Monsell notes, each cognitive task in this mundane sequence of events requires an appropriate configuration of cognitive resources, “a procedural schema or task set” (p. 134). Purposefully switching or shifting task sets involves executive function, but as

Monsell also states, the coordination and selection of task-sets is usually in the service of “our dominant goals” (p. 134) and requires the ability to resist other interests. At once, set shifting is both the production of the appropriate strategy and the inhibition of others – another connection to the theoretical view of Kuhn and Pease (2010). Online Reading Comprehension 19

Arguably, set switching is the muddiest of the three proposed EFs in this study. Why include it in this model, then? If we consider the number of cognitive sets engaged by the online reading comprehension process, the answer becomes clear. Consider a student asked to learn everything she can about snowy owls on the Internet (I remember doing this project myself with a card catalog in grade 4). First, she needs to generate some questions to direct her search. She decides to investigate (1) where snowy owls live, (2) why they’re called snowy owls, (3) and what snowy owls eat. Next, she needs to enter keywords into a search engine. Then, she needs to decide which results are the most relevant and credible. Then, she needs to read the information, evaluate it, decide if it’s useful, and think about what she still needs to know. Just as in

Monsell’s example of the professor, every action in this sequence requires the activation of a schema or set. And, as she continues to learn, she will presumably activate more. She’ll need to activate “executive control” (Monsell, 2003, p. 134) to select and implement the task-sets required to meet her goals. The ability to switch sets is therefore an important EF to include in this model.

Hypotheses

The research question has been stated. To recap, the proposed study will investigate the following question: For a diverse group of typically developing 8th graders, what is the relationship between online reading comprehension ability and executive function while holding constant time spent online for different purposes and offline reading comprehension ability?

Having now reviewed the literature on each of the variables included in the question, I can outline the research hypotheses. Though theory would suggest that measures of executive function are correlated with measures of problem solving processes, online reading comprehension processes included, it is not clear from the existing literature to what degree or in Online Reading Comprehension 20 what direction the correlations might be. Further, it is not at all clear whether the proposed independent variables will account for unique variance in online reading comprehension when measures of time spent online and offline reading comprehension ability are held constant. For this reason, it is hypothesized that individual differences in executive function will correlate with the measure of online reading comprehension ability but I do not know to whether this relationship will be statistically significant. The proposed study should therefore be considered exploratory and design choices, as outlined below, have been made accordingly.

Method

Type of Research Design

The proposed study will use a correlational design to explore the relationships between online reading comprehension scores and three components of executive function: (1) response inhibition, (2) task switching and (3) memory updating, while holding constant (a) students’ scores on Michigan’s 8th grade reading comprehension test and (b) time spent online.

This research design was chosen because it permits for the exploration of individual differences that have occurred naturally in the sample group. It does not require an intervention because the assumption here is that the “interventions” have already occurred. In this case, it is assumed that students have engaged with printed text and online text to different degrees both in school and out of school, that their “typical” cognitive development has been supported by a range of life experiences, including literacy experiences, and that the variability in their experiences will explain, at least in part, the variability across the dependent and independent variables of interest to this study.

The choice to use a correlational design was also taken because the field of online Online Reading Comprehension 21 reading comprehension is emerging. Very little is understood about what makes online reading comprehension different or more complex (Coiro, 2008) than traditional print-based reading comprehension, particularly for young adolescents. This design will allow for preliminary modeling of (a) individual differences in executive control, (b) time spent online and (c) offline reading comprehension – all variables that are hypothesized to contribute to an adolescent’s ability to read for information and understanding online.

Understanding the unique contribution that each independent variable may (or may not) make to the variance in the dependent variable will depend on the construct validity of the instruments used to measure all variables. This is a critical step and in the following discussion of instruments, special care will be taken to emphasize the rationale supporting each choice.

Population of Interest

The population of interest to the proposed study can be defined as typically developing adolescents in the United States. Given the limitations of time and resources for the proposed study, I plan to study a small sample of American teens growing up in Michigan. The sampling frame will be one Michigan middle school.

Sample

A diverse sample of 50 8th grade students who attend a public middle school in Michigan will be recruited to participate in this study. Participants will be randomly selected from a constrained roster of “typically developing” 8th graders in the school. Students with identified learning disabilities will be excluded from the sample because their unique learning needs deserve consideration that is beyond the scope of this preliminary study. Moreover, I would like to understand the relationship between EF and online reading comprehension in a sample (and population) of typically developing students before looking at unique cases. Online Reading Comprehension 22

Given the potential applicability of the findings to new literacies curriculum development for all students, it will be important to include students from a range of cultural, and socio- economic backgrounds. As with any measure of executive control, individual differences may, in fact, be attributable to the cultural, academic and economic conditions in which children have grown up. The purpose of the current study is not to explain the cause of any significant differences in executive function but rather to determine the relationship that may or may not exist between the latent constructs of executive function and online reading comprehension ability.

Recruiting

Principals at middle schools in and around Albion, Michigan will be contacted and informed of the proposed study’s questions and goals. It is hoped that at least one principal will agree to allow the study to take place in his or her school. The middle school that participates in this study will need to provide a quiet place for testing to occur and allow students to miss a block of approximately two hours to participate in the study.

Parental Consent and Student Assent

Parents/guardians of students randomly selected to participate in this study will be sent a letter outlining the study’s purpose and methods. In accordance with IRB guidelines, the letter will ask for their permission to allow their child to participate in the study. The letter will also ask for permission to access and use their child’s MEAP reading score in the study.

Students will also be asked for their assent to participate in the study. This will be important since motivation or lack thereof could be a potential confound in the psychological and online reading comprehension tests. Online Reading Comprehension 23

Instruments

Several instruments will be used in the proposed study to measure the variables of interest. In this section, I will outline each variable of interest and the tests that will be used to measure it. I will also provide a rationale for the selection of each instrument. Table 1 sumarizes all proposed instruments.

I propose using computerized versions of six instruments to measure individual differences on three elements of executive function because of their accuracy in recording response times as well as correct responses and errors. Plus, computerized versions standardize the test-taking experience for every participant. However, as noted below, it will be important for me to understand the role that computerized administration of these tests plays on students’ scores.

Piloting Efforts. One potentially confounding issue, particularly for the tests of executive control, will be the mode of test adminstration. Different results on computerized and non- computerized versions of the tests of executive control may be attributable, in part, to the presence or absence of the computer interface. In order to understand the role that the computer may play in these tests, it will be important to pilot computer and non-computerized versions of at least one of the tests with a group of students matched in every way to the sample population.

This will allow me to assess the costs and benefits of computer vs. non-computerized versions of the tests. Ideally, a random sample of 8th grade students from a school comparable to the participating school for the main study will participate. Students would be randomly assigned to two groups, a computer group and a non-computerized group. Mean scores on the test would be compared to see if the mode of delivery played a significant role. Though I propose computerized versions of each test of executive function for the study itself, evidence collected Online Reading Comprehension 24 through piloting may indicate the need for a change to non-computerized versions of the tests.

At the very least, the pilot comparisons will help me to interpret data collected through computerized testing more accurately.

ORCA-Iditarod (revised). Developed at the University of Connecticut, the Online

Reading Comprehension Assessment (ORCA) – Iditarod (revised) (Leu & Reinking, 2005) is an online assessment tool that measures students’ ability to question, locate, evaluate, synthesize and communicate information on the Internet. The topic of the test is the Iditarod dogsled race.

Students engage with a variety of web-based activities to answer questions about the race and about the websites they use. Screenshots of the ORCA Iditarod –revised and its scoring rubric are included in Appendix A. Construct validity for this test, as measured by the percentage of variance in the mean score explained by the composite score on the test is reported as 41.6%.

The Cronbach alpha for this test, a measure of the test’s internal reliability, is reported to be .73

(New Literacies Research Team, 2009).

A potential criticism of this test is that it constrains student use of the Internet and forces attention to certain elements that, in effect, undermine the ecological validity of the measure.

Though certain activities ask students to read and evaluate prescribed websites, other activities allow students to search the open web to learn about the dogsled race in an ecologically authentic way. The design of the test endeavors to balance concerns of ecological validity with the other concern of acquiring reliable measures across student participants for different new literacies skills. A reliable measure of students’ ability to evaluate websites for accurate information for instance, should present students with the same task. Otherwise, differences in the stimulus might confound the measure. It stands to reason, however, that when students are asked to use the Internet to learn what they can about the dogsled race that they should have full and open Online Reading Comprehension 25 access to the Internet. The design of the test, in my opinion, adequately addresses the trade-offs inherent to the task of quantifying skills that emerge on the Internet.

A second rationale for the selection of the ORCA-Iditarod (revised) over a potentially more ecologically valid navigation of the open Internet pertains to the developmental level of the participants. Constraining the system a little bit restricts the cognitive load of the activity and gives less confident online readers a chance to show what they can do. Essentially, this task should be within the zone of proximal development of all students who participate – and it controls against floor effects, which might also confound a measure of online reading comprehension.

MEAP Reading Score. Eighth-grade reading scores from the Michigan Educational

Assessment Program (MEAP) (Michigan Department of Education, 2009) test will be used to estimate students’ reading comprehension skills. The MEAP test measures grade-level content expectations for reading (Michigan Department of Education, 2006). Students read three passages and answer both multiple choice and short-answer questions. At least one of the reading passages included on the test is informational. The test also evaluates metacognitive skills associated with reading (Michigan Department of Education, 2005).

Example MEAP Reading Comprehension questions and Grade 8 content-level expectations that are assessed by the reading test are included in Appendix B. I acknowledge that standardized tests of reading comprehension are controversial since they typically use a multiple choice format that may, in fact, confound the measure of the intended latent variable

(comprehension ability). Moreover, it is virtually impossible for standardized measures of reading comprehension to capture the true complexity of the reading comprehension process.

That said, the MEAP score has been chosen as an estimate of reading comprehension ability for Online Reading Comprehension 26 several reasons. First, it is administered in students’ schools – an ecologically valid environment because, theoretically, they have both learned about reading in this environment and been tested there. Second, the test is designed to evaluate students’ ability to comprehend a range of texts.

Third, the test is specifically written for 8th grade students in Michigan. The 8th Grade reading curriculum in Michigan serves as the framework for the assessment, which should make the

MEAP test a fair indicator of what students at this level, in this state can read. Finally, it’s a practical choice. Students do this testing anyway so MEAP scores eliminate the need for an additional test. This saves time, money and reduces the demands on the students so that their cognitive resources can be invested in the tests of executive function and online reading comprehension. Online Reading Comprehension and Executive Control 27

Table 1 Summary of proposed variables, instruments, measures and timing

Variable Dependent (D) Instrument Measure Availability Time to Independent (I) administer

Online Reading D ORCA-Iditarod Rubric scores on all five new New Literacies Research Team, University 45 min. Comprehension literacies skills plus composite of Connecticut score.

Offline Reading I 8th grade MEAP Individual score Parent permission required for access. N/A Comprehension reading score

Time Spent I 3 time-diary Average of time on 3 tests (min.) I will develop this. 15 Online surveys min/survey

Inhibition I Stroop Color Word Word reading score I will develop a computerized version but 10-15 min. Test Color naming score the test is available from www.parinc.com Color word score

Inhibition I Stop Signal Test Go Trial Reaction Time (RT) (in Cambridge Cognition www.camcog.com 20 min. ms) Stop Signal RT Stop Signal Delay (in ms) # of direction errors Proportion of successful stops

Updating I CANTAB VSWM 24 measures, most important 3 are Cambridge Cognition 8 min. Test # of errors, a measure of strategy www.camcog.com and a composite measure of response speed (in ms)

Updating I Keep Track Test Proportion of correct words Based on Friedman et al. (2008) I will 10 min. recalled. develop.

Set Shifting I Wisconsin Card Most important three measures: Available from www.parinc.com 15-30 min. Sorting Test # of sequences completed, # of perseverative errors and total # of trials to complete first category.

Set Shifting I Category Switch Switch cost (in ms) Based on Friedman et al. (2008). I will 10 min. Test develop. Online Reading Comprehension and Executive Control 28

Time spent online for different purposes

In their recent study of how college students’ use of the Internet and television impact their reading practices, Mokhtari, Reichard and Gardner (2009) used an innovative time-diary survey technique originally developed by Robinson and Godbey (1997) and adapted by Nie and

Hillygus (2002). Rather than ask students to estimate how much time per week they spend at different tasks, this method breaks the day into six, four-hour blocks and asks students to think of the amount of time in a given block that they spent on a particular task the previous day.

According to Nie and Erbring (Nie & Erbring, 2002) this method is a more accurate way to collect time-use data because it reduces cognitive load. It asks students to think about only one block of time, rather than estimate activity for a whole week. Further, this task is much less demanding than time diaries that ask students to record all of their activities for a week.

The current study proposes an adapted version of this method. The studies cited above had sizable samples but the proposed study will not. Further, these studies were most interested in group-level analysis of time trends. To get an accurate picture of how students use their time, all participants will be asked to fill out three time-diary surveys at randomly selected times over the duration of the data collection phase of the study. The survey, included in Appendix C, will ask students to indicate how much time they spent on the previous day engaged in Internet-based reading for different purposes during the six four-hour blocks of the day. An average of the three scores, measured in minutes/day will be used for analysis. The surveys will take about 10 minutes to complete. Based on the students’ schedules, I will negotiate an appropriate place and time with their teachers for the administration of the survey at school. For weekend days that are randomly selected, I will contact students via text-message, telephone or email (if possible) to ask them to fill out the time diary. Online Reading Comprehension and Executive Control 29

To check for reliability, the data collected with this instrument will be compared with the nationally representative data for this age group from the Kaiser Family Foundation report

(Rideout, et al., 2005).

Measures of Executive Function

As noted by Friedman et al. (2008) understanding the structure of executive functions is complicated by the “so-called task impurity problem” (p. 202). Since executive functions also contribute to other cognitive processes, it is difficult to know how much of the variance in a single executive function task is attributable to the putative function or to the noise caused by other cognitive functions also tapped by the task. For this reason, Friedman et al. (2008) recommend using multiple measures of executive function that theoretically access the executive function of interest but that have different non-executive requirements (e.g., visual or auditory processing). In this way, researchers can statistically extract the shared variance from these tasks to construct latent variables. Generally, however, 200 or more participants are required to use the statistical equations for latent variable analysis. One solution to this problem is partial least squares structural equation modeling, which is outlined below in the section on statistical analyses.

Inhibition

Inhibition Instrument 1: Stroop Test. The Stroop test (Stroop, 1935) is a classic test of inhibitory control that measures the ease with which a person can name colors while inhibiting the pre-potent response to read words (Strauss, et al., 2006, p. 477). The Golden Stroop Word

Test for children aged 5-14 (Golden, Freshwater, & Golden, 2003) is available for $104.00 at http://www3.parinc.com/products/product.aspx?Productid=STROOP-CHILD. The test produces Online Reading Comprehension and Executive Control 30 three scores: (1) The word-reading score. This is the number of color words read correctly in 45 seconds. Participants read from a page on which 100 color words (i.e. red, green, blue) are printed in black ink. (2) The color-naming score. This is the number of colors correctly named in

45 seconds. On this page, 100 Xs are printed in red, green or blue ink. (3) The color-word score.

It is the number of colors named in 45 seconds. For this task, participants read the same words presented in the word-reading task but this time, the words are printed in ink colors that are inconsistent with the meaning of the word. For children, an interference score is also calculated by finding the difference between the color-word T-score (M=50, SD=10) and the color T-score.

In a sample of 7th graders, Protopapas et al (2007) found that Stroop interference was lower for better readers, suggesting a relationship between reading ability and the ability to inhibit.

Like most tests of executive function, performance on the Stroop test improves with practice. On the Golden version for adults, test-re-test reliability has been found to be 0.83 for the Word score, 0.74 and for the Color score and 0.67 for the Color-Word score (Franzen,

Tishelman, Sharp, & Friedman, 1987). Golden and colleagues have not provided test-retest reliability data for the children’s version of the test.

Correlations among the three test tasks (Word, Color and Color-Word) tend to be moderate/high for normal individuals (Strauss, et al., 2006, p. 492). In a study that compared a control group of typical undergraduates with a sample of adult patients receiving neuropsychological treatment for a variety of issues, Chafetz and Matthews (2004) found that for the control group, the r for word-color scores was .557, the r for word-color/word scores was .

494 and the r for color-color/word scores was .591. Since the three tests are not perfectly collinear, we can assume they test unique underlying mechanisms.

Friedman and Miyake (2004) found the Stroop test to be significantly but weakly Online Reading Comprehension and Executive Control 31 correlated with a stop-signal test (r =.15), another measure of inhibition, proposed below as the second measure of inhibition. Given the low correlation between them, it is assumed that these two tests engage different aspects of the latent inhibitory variable.

If appropriate based on piloting data, a computerized version of the Golden Stroop Task for Children will be developed using PsyScope, an open-source software for developing psychological tests on a Mac OS (Cohen, MacWhinney, Flatt, & Provost, 1993). The test will be an adapted version of the Stroop test used by Friedman et al. (2008, p. 205).

Inhibition Instrument 2: Stop Signal Task. Friedman et al. (2008) have used a stop-signal task to measure response inhibition. Generally, stop-signal tests train a pre-potent response by asking participants to repeatedly follow a rule in the first phase of the test (e.g. categorize these words as animal or non-animal as quickly as you can; press the left arrow key when you see left- pointing arrow and the right arrow key when you see a right-pointing arrow). The next phase introduces as new rule that is associated with the presentation of a signal, usually an auditory tone (e.g. when you hear a beep, don’t press either arrow). As Logan (1994) outlines, the stop- signal paradigm includes three essential measures: (1) the amount of time it takes for a participant to respond to the stimulus in a “go” trial during the training phase, called the go- signal reaction time, (2) the amount of time that elapses between the presentation of the stop signal and an incorrect response, called the stop-signal reaction time and (3) the stop-signal delay which is the amount of time that people wait to see if the stop-signal will be presented before they respond during the second phase of the test. Logan (1994, p. 223) notes that people will delay their reaction times to the initial stimulus as a function of the probability of the stop signal.

If they feel there is a high likelihood of a signal, they’re more likely to inhibit their response. It is therefore important to present the stop signal only 25% of the time so that participants don’t Online Reading Comprehension and Executive Control 32 expect it and don’t change their reaction times on go trials so significantly. It is also important to tell participants to always answer as quickly as they can to the initial stimulus (Logan, 1994).

Cambridge Cognition (www.camcog.com) sells a computerized version of the stop-signal test with arrows. The test produces the three measures outlined above plus number of direction errors, and the proportion of successful stops. Friedman et al (2008, p. 208) report internal reliability as .76 for the version they used in this study and .72 for another version used in another study (Friedman & Miyake, 2004, p. 111). The internal reliability coefficient for the

Cambridge Cognition version of the stop-signal test is not available at their website.

Updating Working Memory

Updating Working Memory Instrument 1: The CANTAB Spatial Working Memory Test.

This test measures the ability to retain spatial information and manipulate remembered items in working memory. It is available from Cambridge Cognition (www.camcog.com). The test begins with colored squares shown on a screen. The aim of the test is to fill an empty column, which appears to the right of the screen, with blue tokens. The tokens appear by clicking on the colored squares. Sometimes, when a participant clicks on a colored square, a token appears; sometimes, there is no token. However, participants are told that tokens will only appear under a colored square once per trial. This means that the participant has to remember which squares he or she has clicked already and under which boxes there were tokens. This allows the participant to gradually refine the search process for tokens. The number of colored boxes increases gradually to a maximum of eight boxes. Color and position of boxes changes from trial to trial to discourage use of stereotyped search strategies. The CANTAB version of the test delivers 24 outcome measures including the number of errors, a strategy measure and latency (speed of subject’s responses) measures. Online Reading Comprehension and Executive Control 33

According to Strauss, Sherman and Spreen (Strauss, et al., 2006, p. 415) this test is easy to administer to children older than nine years of age. It takes approximately eight minutes to complete. Test-retest reliability for the strategy measure on the CANTAB Spatial-Working-

Memory test was found to be .63 for adults with a standard error of probability (SEP) of 3.86

(Cambridge Cognition Limited, 2008). For the measure of errors, the test has a test-retest reliability coefficient of .70 and an SEP of 12.54 (p. 3).

Updating Working Memory Task 2: Keep Track Task, adapted from Friedman et al.

(2008). The proposed keep-track task will follow Friedman and colleagues’ (2008, p. 206) procedure. In their study, participants were first shown several target categories at the bottom of a computer screen. There were six possible categories (animals, colors, countries, distances, metals and relatives) but for any given trial, only a few of them were presented on the screen.

Fifteen words, including two or three exemplars from each of the selected categories were then presented serially and in random order in the center of the screen for 1, 500 ms each. The target categories always remained at the bottom of the screen. The task was to remember the last word presented in each of the target categories and then report these words at the end of the trial. The outcome measure was the proportion of words recalled correctly. Construct validity and test-re- test reliability data are not available for this test but Friedman et al. (2008, p. 223) report that results on the keep-track test correlated to varying degrees with other measures of WM (r=.45 for a letter memory task and r=.29 for a spatial two-back task), inhibition (r=.22 with the Stroop task) and set shifting (r=.17 on the category switch task, also proposed for this study).

It is important to note that I am proposing to use a spatial WM and a word-based WM task in the proposed study to collect data on two different aspects of WM as proposed by Online Reading Comprehension and Executive Control 34

Baddeley (2007). This should help me to address the “task impurity problem” (Friedman, et al.,

2008).

Set Shifting

Set Shifting Instrument 1: Wisconsin Card Sorting Test (WSCT). The Wisconsin Card

Sort Test (WCST) (Berg, 1948; Grant & Berg, 1948) is a classic test of set shifting. As summarized by Strauss et al. (2006, p. 527) the test consists of four stimulus cards, placed in front of the participant. The first card has a red triangle on it, the second, two green stars, the third, three yellow crosses and the fourth, four blue circles. The participant is given two stacks of

64 cards. These cards have designs on them that are similar to the four stimulus cards. The participant is told to match each card from the deck with the stimulus card that it most resembles.

With each match, the participant is given feedback about whether the match is correct or incorrect. The trick is that the “rule” governing the matching changes through the test, but the participant is not told the rule has changed. Participants must first sort by color, then by form, then back to color and then by number, but they have to devise these set switches themselves by using the feedback provided by the interviewer. This procedure continues until the participant has successfully completed six sorting categories of 10 cardd each or until all 128 cards have been used (Strauss, et al., 2006, p. 528). Measures include (1) the number of sequences of 10 consecutive correct matches (max=6), (2) total number of trials to complete first category, (3) the number of perseverative errors, which is the number of times a participant persists in responding to a stimulus characteristic that is incorrect, (4) percent of perseverative errors in relation to overall test performance, (5) failure to maintain set, which is when a participant makes five or more correct matches but then makes an error before successfully completing the category, (6) percent conceptual level responses which is the number of runs of three consecutive correct Online Reading Comprehension and Executive Control 35 responses, and, (7) Learning to learn, which is the participant’s average change in conceptual efficiency across the successive categories. The test takes between 15 and 30 minutes to complete and is appropriate for children aged 5 and older. For children and teens, performance on the WCST improves with age (Chelune & Baer, 1986) and stabilizes during mid-adulthood.

Performance tends to decline, however, for older adults (Rhodes, 2004). Paniak, Miller, Murphy,

Patterson and Keizer (1996) have published normative data for children aged 9-14 (see summary in Strauss, et al., 2006, pp. 531-532).

As with other tests of executive function, measures of test-retest reliability on the WCST are low since it is expected that people remember the nature of the category switches even after time has elapsed (Strauss, et al., 2006). For percent perseverative errors, for instance, Heaton and colleagues (Heaton, Chelune, Talley, Kay, & Curtis, 1993) report a generalizability coefficient of

.37 for healthy children and adolescents, suggesting that performance on the second test will probably be different than performance on the first. Moreover, for typically developing children, the standard error of measurement is estimated to be about 10.28 for perseverative responses

(Heaton, et al., 1993), suggesting substantial variability within the population. Fortunately, the proposed study doesn’t intend to re-test participants with the WCST or any test of EF.

The WCST has been criticized as a task too complicated to generate a pure measure of set-shifting, however (Best, et al., 2009; Miyake, et al., 2000; Zelzao, et al., 2004). The test’s purpose, as defined by Strauss et al. (see 2006, pp. 531-532 for a summary) seems to make this point. They note, “the purpose of this test is to assess the ability to form abstract concepts, to shift and maintain set, and to utilize feedback” (p. 526). And yet, as Monsell’s review suggests

(2003) set switching, though considered a foundational EF, probably isn’t a singularly unified construct. Rather, abstraction, and utilizing feedback to mediate or drive the task switch are Online Reading Comprehension and Executive Control 36 essential to the process. The purposeful act of switching is driven by a goal. So, while this measure may include variance attributable to other elements of task switching, the whole process itself is of interest to the current study. On the Internet, nobody tells a teenager where to look or how to interpret this website vs. the next one, though different sets may be required for their interpretation. For this reason, the WCST has been included in the proposed study.

Set Shifting Instrument 2: Category Switch Task with a Cue, adapted from Friedman et al., 2008. This test, developed by Friedman et al. (2008, p. 207) asked participants to categorize words as (1) a thing that is either living or non-living or (2) a thing that is smaller or larger than a soccer ball. Each trial included 16 nouns, each of which could be categorized on these two criteria (e.g., table, bicycle, coat, pebble, cup, marble, horse, shark, mouse, sparrow etc.). A symbol appeared above each word and cued the participants about which category to use. A heart indicated the participants were to use the living/non-living category and an arrow cross indicated large vs. small. Participants completed four blocks of 48 trials, each of which contained 24 switch and 24 no-switch trials. Each trial was preceded by the cue (heart or cross), though on the first and third blocks the cue was presented 150 milliseconds (ms) before the word whereas on the second and fourth blocks, the cue was presented 1500 ms before the word. Participants were asked to use whatever time they had to prepare for the next stimulus and were told to respond as quickly as they could. The outcome measure for this task was the switch cost, calculated as the difference between average responses times (RTs) of the trials that required a switch and the average RT on trials that required no category switch when there was 150 ms before the word appeared on the screen. Friedman et al. (2008) report internal reliability for this measure as .85

(p. 208). Online Reading Comprehension and Executive Control 37

As a second measure of set-switching, this task is simpler than the WCST because the category shift is cued in advance. Participants don’t need to learn a new rule based on feedback which, ostensibly, removes the potential confounds of “abstraction” and “learning from feedback”. Together, however, these two measures should provide a solid indication of set- switching ability in this sample group.

Statistical Anlaysis

The hypothesized relationships between the dependent and independent variables in the proposed statistical model are shown in Figure 1.

Figure 1. Structural equation model of factors contributing to online reading comprehension

I 1

I 2

TDM Time EF U Online Reading 1 Comprehension

Offline Reading U 2 MEAP Comprehension

S 1

S 2

Note.TDM = Time Diary Mean Score, MEAP = Standardized Reading Comprehension Score, EF = Executive Function, I1=Inhibition Task 1, Stroop Test, I2=Inhibition Task 2, Stop-Signal Task, U1=Updating Working Memory Task 1, Cantab Visual Spatial Working Memory Task, U2=Updating Task 2, Keep Track Task, S1 = Set Shifting Task 1 = Wisconsin Card Sorting Task, S2=Set Shifting Task 2, Category Switch Task. Online Reading Comprehension and Executive Control 38

Partial least squares structural equation modeling will be used to generate a statistical model of the relationships between the dependent and independent variables in the proposed study. As noted by Chin and Newsted (1999, p. 312) partial least squares structural equation modeling (PLS) is an alternative to traditional covariance-based structural equation modeling

(CBSEM). PLS is used to generate predictive, rather than causal claims about the relationships between the latent variables (p. 312). Latent variables are defined as the sum of their respective indicators and the PLS algorithm “attempts to obtain the best weight estimates for each block of indicators corresponding to each latent variable.” (p. 312). Importantly, PLS models allow for the inclusion of both reflective and formative indicators; reflective indicators being those ostensibly “caused” by the latent variable and formative indicators being those assumed to cause the latent variable (Chin & Newsted, 1999, p. 310). Moreover, formative indicators do not have to be independent from one another. In this case, the indicators of Inhibition, Updating Working

Memory and Set Shifting are taken as distinct but unified indicators of the overarching latent variable – executive function. These indicators are therefore viewed as formative. Moreover, time and offline reading comprehension are considered formative indictors of online reading comprehension, though there could be some sort of bi-directionality inherent to their relationships. It also seems the most logical to view EF as a formative indicator of online reading comprehension. As a general cognitive function, it seems more likely that EF would be a cause of online reading comprehension than the other way around.

Chin and Newstead (1999, p. 337) suggest that PLS is appropriate for studies in the social sciences where (1) the objective is prediction, (2) the phenomenon in question is relatively new or changing and the theoretical frameworks are emerging, (3) the model is relatively complex with large numbers of indicators, (4) there is an epistemic need to model the relationships Online Reading Comprehension and Executive Control 39 between the latent variables and indicators as either formative, reflective or both and (5) the data conditions relating to normal distribution, independence or sample size are not met. Though the proposed model would not be considered overly complex (condition 3), the other four conditions listed above describe exactly the realities of the proposed study. Due to time and funding limitations for the proposed study, only a small sample size is realistic. Though CBSEM is preferable for making causal inferences, PLS is able to generate reliable analyses of the relationships between the variables for “soft modeling” (Chin & Newsted, 1999, p. 336), i.e. when description and prediction are the only goals of the research design.

Power Analysis

Chin and Newstead (1999, p. 314) note that in PLS, power analysis is based on the portion of the model with the largest number of predictors and that sample size is calculated by using a regression heuristic of 10 cases per predictor. In the proposed model, EF has the largest number of predictors (6) meaning that the sample size should be approximately 60. That said,

Chin and Newstead also note that sample size and power are dependent on effect size. If the correlations in this part of the model are larger, the sample size could decrease. To estimate the correlations, I’ve used a study by Cain, Oakhill and Bryant (2004) that found a statistically significant correlation, r = .498,  = .001, between their measure of working memory and reading comprehension. Using Sloper’s post-hoc statistical power calculator for multiple regression (Sloper, 2009) I calculated the expected power to be 0.828 with a sample size of 50, six predictors in the largest part of the regression model and a theoretical R2=.25. Online Reading Comprehension and Executive Control 40

Practical Significance

The practical significance of the proposed study relates to the potential contribution it could make to the development of teaching practice and curricula for new literacies. If subcomponents of executive function do indeed contribute unique variance to the dependent measure, subsequent research might turn to the investigation of teaching practices that support the development of executive function for reading. Further, if executive function is found to be significantly correlated to online reading comprehension, this research could lead to the equally important investigation of the developmental trajectory of executive function for online reading comprehension and how it differs from the development of offline reading comprehension skills.

Ethical Issues

Since I am proposing to conduct psychometric tests of executive function on teens in this study, results may reveal previously undiagnosed problems with executive function in one or more participants. Should this occur, I would feel a responsibility to discuss the results with the student’s guidance counselor, making it clear that I am not a licensed clinician or a neuropsychologist and cannot therefore make any kind of clinical diagnosis. With the counselor,

I would decide whether it would be appropriate to contact the child and his or her parents with the testing information. With parents, I would also emphasize that I am not a licensed clinician or a doctor but I would explain that I felt an obligation to share the test results with them so that they could pursue further assessment if they so choose. I would also emphasize that the testing was not done in a clinical environment or with clinical objectives in mind so that different results may emerge in a different situation and with a different interviewer.

Participation in this study will also mean that students will miss approximately three hours of instructional time. Though the activities in this study will engage and challenge Online Reading Comprehension and Executive Control 41 students, it will be important to schedule student participation at times when they will not miss critical lessons and evaluations. This will require negotiation with teachers. I also acknowledge that when students miss lessons – critical or not – it creates work for teachers who must then support the students in catching up. I will address this ethical issue in the following ways. First, I propose to donate (amount to be determined) to the school library in acknowledgement of the generous contribution the school community made to this study. Second, I will give the school administration, teachers and the students plenty of advance notice of the study schedule so that they might be pro-active in addressing the issue of “catch up” but so that they might also tell me when students cannot participate. I will encourage student participants to make arrangements with their teachers in advance of their “interview” time so that work might be completed in advance rather than after the fact. I will also encourage students to ask a friend to take notes during the missed classes.

Summary

The proposed correlational study aims to extend our current understanding of online reading comprehension by framing it as a complex, strategic, problem solving process mediated by executive function (Kuhn & Pease, 2010). Based on existing theory, it hypothesizes that for typically developing adolescents, executive function, and more specifically three foundational

EFs: response inhibition, updating working memory and set-switching may be related to online reading comprehension if offline reading comprehension and time spent online are held constant.

This study is important because it stands to offer much needed empirical evidence in response to theories (Coiro, 2007; Duke, Schmar-Dobler and Zhang, 2006; Hartman, Morsink & Zheng, in press) that suggest online reading comprehension makes additional cognitive demands of the reader. If significant, data about the relationships could form the basis for future study on the Online Reading Comprehension and Executive Control 42 cognitive developmental trajectories of online reading comprehension and on the design of new literacies curricula that support the development of foundational EFs for online reading. Online Reading Comprehension and Executive Control 43

References

Appendix A

Screen Shots of Existing ORCA Open Assessment (ORCA-Iditarod Revised 2007-08) (New Literacies Research Team, 2009)

Task Introduction

Students are introduced to the online reading task in a series of screens. Pictured here are the initial set of task directions (Figure A) – read aloud in an audio file for students who struggle with decoding – and an excerpt from the Iditarod text (Figure B) provided in the ORCA-Open system (embedded with audio and hyperlinked information). This text was created to help students build background knowledge about the topic before they begin to use their online reading comprehension skills to locate, evaluate, synthesize, and communicate the requested information.

A B Online Reading Comprehension and Executive Control 44

ORCA-Open Protocol: During the ORCA-Open protocol, D students toggle between task directions presented in a quiz interface (Figure C) and the unbound Internet where they interact with real search engines (Figure D) and myriad online websites (Figure E) as they search for and locate relevant information. Students who are able to locate the relevant website(s) then return to the quiz interface (Figure C) to respond to a range of open and short answer prompts. In this case, students are asked to read the website critically to determine the author’s purpose for writing the information and then provide evidence that supports their decision. E

C

Using Online Communication Tools

Within the ORCA-Open interface, students are asked to read and post their own information using online communication tools such as blogs, wikis, discussion boards, and/or email. Pictured here is a screenshot of the online discussion board created in the ORCA-Open quiz interface (Figure F) to simulate a conversation between several people who have different opinions of the Iditarod dog sled races. Students navigate the discussions and then synthesize and communicate their own opinions (based on the websites and discussion postings they read) in an email message (see Figure G). Students are then sent outside the interface to send their email message, with text that gets scored as an open response. F G Online Reading Comprehension and Executive Control 45

\

Rubric for ORCA-Open

Student responses are scored using a rubric scoring system. Figure H shows the scoring system for responses to tasks for locating and critically evaluating the accuracy of online information. Figure I shows the scoring system for critically evaluating the relevancy and reliability of online information.

H I

Appendix B English Language Arts Grade 8 Reading Expectations, v. 12.05 and sample reading comprehension questions from the 2005 and 2006 8th Grade MEAP reading comprehension test (Michigan Department of Education, 2006) Online Reading Comprehension and Executive Control 46 Online Reading Comprehension and Executive Control 47 Online Reading Comprehension and Executive Control 48

Released items from 2005 Grade 8 Reading Test: King Alfred and the Cakes

1. List three details to show why King Alfred could be described as “Alfred the Great.”

2. Use three details to describe how King Alfred felt about being a king.

3. Choose one word to describe King Alfred. Use two details from the text to support your answer.

Released items from 2006 Grade 8 Reading Test: Mysid Shrimp: Testing for Poisonous Garbage

1. Use three details to describe how the shrimp react after eating the garbage.

2. Describe mysid shrimp using three details from the text.

3. Explain how mysid shrimp are used to test poisonous garbage using three details from the selection. Online Reading Comprehension and Executive Control 49

Appendix C

Time Diary Name: Date: The purpose of this time diary is to understand what you did at different times of the day yesterday. For the blocks of time outlined here, please write in your most accurate description of what you were doing. Examples have been provided so you know what kind of descriptions you might provide. If you did several things during the two-hour block, list the events in order and estimate how much time, in minutes, you spent at each activity.

Time Blocks Minutes What did you do? Where were Who was you? there? 12:00 to 2:00 AM 120 Slept Home

2:00 to 4:00 AM 120 Slept Home

4:00 to 6:00 AM 120 Slept Home

6:00 to 8:00 AM 60 min. Get ready for school - shower, dress, eat Home Mom, dad, breakfast, brush teeth sister

10 min. Watch TV Home Sister

30 min. Walk to school outside friend

20 min. Check e-mail, play web games School library friend

8:00 to 10:00 AM Online Reading Comprehension and Executive Control 50

Time Block Minutes What did you do? Where were Who was you? there? 10:00 AM to 12:00 PM

12:00 to 2:00 PM

2:00 to 4:00 PM

4:00 to 6:00 PM

6:00 to 8:00 PM Online Reading Comprehension and Executive Control 51

Time Block Minutes What did you do? Where were Who was you? there? 8:00 to 10:00 PM

10:00 PM to 12:00 AM