The Deliberate Practice of Procedural Nursing Skills: Effects of Block-Random Sequencing on Long-Term Retention

THE DELIBERATE PRACTICE OF PROCEDURAL NURSING SKILLS: EFFECTS OF BLOCK-RANDOM SEQUENCING ON LONG-TERM RETENTION

A dissertation submitted to Kent State University in partial fulﬁllment of the requirements for the degree of Doctor of Philosophy

Andrew Jude Cerniglia

November 20, 2019 © Copyright, 2019 by Andrew Jude Cerniglia

ii Dissertation written by

Andrew Jude Cerniglia

B.S., Mount Union College, 2000

M.S., Indiana Wesleyan University, 2004

Ph.D., Kent State University, 2019

Approved by

, Chair, Doctoral Dissertation Committee Brad Morris

, Member, Doctoral Dissertation Committee Chris Was

, Member, Doctoral Dissertation Committee Albert Ingram

Accepted by

, Director, School of Lifespan Development and Mary Dellman-Jenkins Educational Sciences

, Dean, College of Education, Health and James C. Hannon Human Services

iii CERNIGLIA, ANDREW J., Ph.D., December, 2019 Educational Psychology

THE DELIBERATE PRACTICE OF PROCEDURAL NURSING SKILLS: EF- FECTS OF BLOCK-RANDOM SEQUENCING ON LONG-TERM RETENTION (79 pp.)

Director of Dissertation: Bradley J. Morris, Ph.D.

The primary purpose of the study was to determine whether the block randomization of deliberately-practiced nursing skills results in an increase in long- term retention. The study utilized a pre-test post-test, within-subjects design. Stu- dent performance was assessed prior to and immediately after training. Sterile dressing change and venipuncture skills were practiced, with each subject practicing one of the skills procedurally and the other skill in block-random fashion. A practical measure of long-term retention was administered three weeks after training. A total of 46 subjects began the study. However, absences on training days and attrition from second to third quarter of the program depressed participation. Thirty adult education LPN students at a small, midwestern vocational school, ranging in ages from 20 to 58 and averaging 30 years of age completed all components of the study. Thirteen of the 30 subjects had completed varying degrees of post-secondary coursework. Results failed to demonstrate significant differences for practice condition and the interaction of time and practice condition. A main effect for time was realized for both skills. The pre-announced nature of the measure of long-term retention may have obscured additional effects. A more tightly controlled study, utilizing applied pre- and post-assessments, in addition to an unannounced measure of long-term retention could answer these questions more definitively. A lab of programmable, high fidelity simulators would allow for the automation and precise control of such a study. ACKNOWLEDGEMENTS

I found this process to be a reminder of how inconsequential we are as individual actors in the word. I will be forever grateful for the patience, understanding, guidance, and support of my advisors Dr. Morris, Dr. Ingram, and Dr. Was. I want to especially thank the nursing faculty who spent many hours accomadating my requests, modifying their curriculum, and altering their work schedules to ﬁt the requirements of the study.

Finally, to my wife and parents who oﬀered their support but always couched in the expectation I would successfully complete this journey, thank you.

iv DEDICATION

This composition is dedicated to my ﬁve children. May they value work, ﬁnd their

passion, and love.

v TABLE OF CONTENTS

ACKNOWLEDGEMENTS ...... iv

DEDICATION ...... v

LIST OF FIGURES ...... viii

LIST OF TABLES ...... ix

I INTRODUCTION ...... 1

II REVIEW OF THE LITERATURE ...... 9

Optimizing Performance During Acquisition ...... 9

Improving Performance After a Delay ...... 14

Summary ...... 22

III METHODS ...... 27

Participants ...... 27

Materials ...... 28

Procedures ...... 29

Coding ...... 31

vi IV ANALYSIS OF THE FINDINGS ...... 35

Sterile Dressing Change ...... 35

Venipuncture ...... 37

V DISCUSSION, LIMITATIONS, AND RECOMMENDATIONS ...... 38

Limitations ...... 40

Recommendations ...... 43

APPENDICES ...... 45

APPENDIX A: VENIPUNCTURE CHECKLISTS ...... 46

APPENDIX B: STERILE DRESSING CHANGE CHECKLISTS ...... 52

REFERENCES ...... 56

vii LIST OF FIGURES

1 Example Study Sequences ...... 32

2 Study Design ...... 33

3 Subject performance: sterile dressing change ...... 36

4 Subject performance: venipuncture ...... 37

viii LIST OF TABLES

1 Within-Subjects Design ...... 30

ix 1

CHAPTER I

INTRODUCTION

Medical professionals are tasked with making decisions in real time based on an analysis of their patients. Emergency room physicians are an ideal example of this requirement, but even the nurse aide is required to make minor decisions regarding patient care during each shift. Complicating matters is the number of variables, sometimes confounding, that must be considered, and the fact that some decisions are related to conditions or circumstances encountered only periodically. In order to successfully navigate this complex problem space, medical professionals employ analytic and non-analytic systems to integrate background knowledge with information collected in real time through inter- actions with patients (Eva, 2005). The combination of these two systems, an intuitive approach also referred to as “System 2” and “System 1”(Kahneman, 2011), has been shown to be superior to either employed in isolation, even for novices (Eva, 2005).

While medical doctors diagnose conditions, prescribe treatment, and perform procedures, nurses are the primary practitioners of day-to-day patient care. They serve as intermediaries, interacting with patients frequently, assessing their condition, incorpo- rating information gleaned through dialogue, all while demonstrating empathy and an understanding of the patient’s condition. As a result of this complexity, nurse education 2 programs necessarily address the cognitive, psychomotor, and aﬀective domains (Nehring

& Lashley, 2008). Instruction takes place in a variety of settings, including the traditional classroom, in the lab setting, and in the ﬁeld. Educators seek to utilize this combination to integrate a comprehensive understanding of human disease with associated practical skills and an empathetic approach to human care.

One dilemma facing nurse educators is deciding how and when to initiation practice on real patients. Fidelity is highest in the clinical setting, however stakes are high. Adequate supervision and prior training are critical. Moreover, while basic everyday nursing skills are practiced later in programs in the clinical setting, the more advanced skills may not be available for practice based on the group of patients admitted to a clinical site when students are completing their rotation. This reality, coupled with the fact that students and professionals lack consistent access to feedback and opportunities for practice aimed at the correction of errors has made simulated laboratory experiences the chosen option for improvement Issenberg, McGaghie, and Hart (1999).

Laboratory experiences provide a safe environment for beginning the process of integrating knowledge and skills to the degree required for graduation to the clinical setting.

As Kaakinen and Ellyn (2009) notes, it is quite possible for a student to perform a skill or a sequence of skills correctly, but mindlessly, i.e., without an understanding of the underlying rationale for their actions. The laboratory environment seeks to extinguish this possibility by allowing for repetition and reﬂection. The instructor monitors performance 3 and assesses understanding, improving both over time. Laboratory simulations attempt to expose students to realistic situations in a safe environment as they practice clinical decision-making skills, communication, and teamwork (Wilford, 2006). More precisely:

“Simulations are deﬁned as activities that mimic the reality of a clinical

environment and are designed to demonstrate procedures, decision-making

and critical thinking . . . A simulation may be very detailed and closely simu-

late reality, or it can be a group of components that are combined to provide

some semblance of reality” (Fawcett, 1992, p. 97).

The range of options available to educators is broad, with high ﬁdelity options often the most expensive. From low to high ﬁdelity, options include anatomical models

(physical models of the human body), task trainers (equipment that allow students to train a speciﬁc skill), role playing, games, computer-assisted instruction, standardized patients (someone who has been trained to portray, in a consistent, standardized manner, a patient in a medical situation), virtual reality, low- and high-ﬁdelity mannequins

(computerized human models; diﬀerentiated by the functionality and the degree to which they are able to approximate real life) (Nehring & Lashley, 2008).

High-ﬁdelity simulations are viewed positively by students, who suggest the experiences boost conﬁdence, and improve clinical and assessment skills (Nehring & Lashley,

2008). Simulations have been shown to improve student performance (McGaghie, Is- senberg, Petrusa, & Scalese, 2006; Issenberg, McGaghie, Petrusa, Lee Gordon, & Scalese, 4

2009), in areas ranging from Advanced Cardiac Life Support (ALCS) Wayne et al. (2005) to laprascopic suturing (Van Sickle et al., 2008). However, some warn the complexity of high ﬁdelity simulations increases the risk of cognitive overload Haji et al. (2016).

In order to further facilitate mindful practice, simulations have been combined with the tenets of deliberate practice Ericsson (2004); Ericsson, Whyte, and Ward (2007).

Deliberately practiced simulation activities are to begin with clearly stated objectives, are to include the provision of cues to ensure missteps are not taken or are corrected quickly, and timely feedback. Sessions are to end with a reflective debriefing followed by the opportunity to improve performance (Jeffries, 2005). Simulations paired with deliberate practice are viewed favorably and have been shown to be effective. A 2011 meta-analysis of this work suggests the implementation of deliberate practice in clinical education has been successful, finding an overall effect size of 0.71 for the 14 studies that met the authors’ requirements (McGaghie, Issenberg, Cohen, Barsuk, & Wayne, 2011).

Over the long-term, however, researchers note that ﬁndings related to the use of simulations are mixed (Nehring & Lashley, 2008). Liou, Chang, Tsai, and Cheng (2013b) note medical administrators and graduating nurses themselves report a lack of preparation for the work place. Bogossian et al. (2014) found only one in ten nursing students were able to successfully respond to a deteriorating patient. Casey, Fink, Krugman, and Propst

(2004) found only 4% of surveyed graduates to be comfortable performing the entire 5 range of nursing skills. 63% of nurse preceptors believe new nurses require more assistance than expected when performing nursing skills (Hickey, 2009). Only 25% of nurse leaders and administrators report they were fully satisﬁed with new nurse performance

(Berkow, Virkstis, Stewart, & Conway, 2008). And, while there was great variability in the responses for most skills, applied skills were consistently ranked as “poor” by the respondents. Moreover, it is unclear on-the-job experience improves performance. Carlisle,

Luker, Davies, Stilwell, and Wilson (1999) found 41% of new graduates were still uncomfortable with some skills after a full year in the workplace, and Bjørk and Kirkevold

(1999) found stagnation and even decline in execution of skills when considering both the sequence and accuracy of performance.

The disconnect between the promising results attributed to the deliberate practice of procedural nursing skills and the feedback from those familiar with the capabilities of new nurses is striking. A close examination of the studies supporting the application of deliberate practice to simulation sessions suggests a somewhat inconsistent application as some studies equate deliberate practice and mastery learning (Issenberg et al., 2002;

Wayne et al., 2006; Barsuk, Ahya, Cohen, McGaghie, & Wayne, 2009), and therefore include only some components, e.g., iterative practice, into the study’s design. More importantly, many studies include only short delays between treatment and assessment

(Issenberg et al., 2002; Korndorﬀer, Dunne, et al., 2005; Korndorﬀer, Hayes, et al., 2005;

Stefanidis et al., 2006; Wayne et al., 2006; Andreatta et al., 2006), or vary the length of 6 delay for control and treatment groups (Wayne et al., 2005; Ahlberg et al., 2007; Wayne,

Didwania, et al., 2008; Barsuk, Ahya, et al., 2009). Generally, the assumption appears to be that nursing skills will be mastered “on the job”, and that the traditional skills check performed once after a relatively brief period of practice is suﬃcient evidence of instructional success. However, a meta-analysis of skill delay and retention completed by

Arthur Jr., Bennett Jr., Stanush, and McNelly (1998) suggests there is substantial skill loss associated with periods of disuse. How might nurse educators improve the long-term retention of procedural nursing skills to account for the fact that subsequent skill use may be intermittent or non-existent throughout the remainder of a nursing program?

One area of research dedicated to improving long-term retention is desirable diﬃculties

(Bjork, 1994). This body of work diﬀerentiates “performance” and “learning”. The former is related to learner performance during or soon after acquisition, whereas the latter represents more durable and accessible representations of content and/or skills.

Importantly, Bjork and Bjork (2009) suggests it is quite possible to realize increases in performance without aﬀecting, or at least without guaranteeing, learning, an increase often associated with a “false sense of knowing”, i.e., a gross overestimation of mastery by both the student and instructor (Agarwal, Karpicke, Kang, Roediger, & McDermott,

2008). Bjork identiﬁes a variety of strategies that introduce diﬃculty during the learning process. These strategies are “desirable” in that they tend to depress performance during acquisition, but improve performance after delays and on measures of transfer. One such 7 strategy is varying the conditions of practice, referred to as contextual interference.

Contextual interference may be introduced by varying the context within which practice occurs, e.g., the physical environment or modality through which practice in completed. Contextual interference may also be obtained by randomizing or “block- randomizing” a sequence of practice, e.g., practicing basketball set shots from varying distances and angles in random or blocked-random order (Landin, 1997). Block- randomization is achieved by separating a logical sequence, or any sequence based on the decomposition of the topic to allow for pre-requisite knowledge to be mastered prior to the introduction of superordinate concepts dependent on their understanding, into three or more “blocks”. The order within each block retains its logical structure, but the blocks themselves are presented in random order. In order to increase contextual interference, blocks from two or more logical sequences maybe interpolated randomly. For procedural skills, logical sequences are based on the order in which the sequence of skills must be completed to ensure compliance and/or safety.

The medical community has embraced the training strategy of coupling deliberate practice and simulation in the laboratory setting. Research suggests these efforts are successful in the near-term. However, feedback from nurse administrators indicates a lack of satisfaction with the skill level of new nurses. While nurses are typically required to “check off” on skills prior to entering the clinical setting to assure a basic level of 8 competence, periods of disuse appear to diminish skill levels significantly. Taken collectively, this body of work suggests seeking strategies to improve long-term retention, for the purposes of this study defined as assessments that take place a minimum of two weeks after training has been completed, are warranted. Varying the conditions of practice via block-randomization, a desirable difficulty, is one such strategy. 9

CHAPTER II

REVIEW OF THE LITERATURE

Optimizing Performance During Acquisition

Optimizing performance during or shortly after acquisition is dependent on an in- depth analysis of content and learner, resulting in the creation of a logically sequenced, scaffolded design which incorporates practice opportunities that stretch learners but en- sures they have available the supports necessary to assimilate content in an efficient manner. Van Patten, Chao, and Reigeluth identify two steps required to construct a sequence; the identification of the elements to be sequenced, and the selection of the organizing principle or relationships between elements of the content. Content may lend itself to chronological sequencing (Mager, 1961), sequencing based on world relations, concept relations, inquiry relations, or utilization relations (Posner & Strike, 1976).

Additional sequencing methods include simple-to-complex (Wilson & Cole, 1992) and general-to-speciﬁc. In traditional designs, prerequisite information is viewed as the necessary building blocks of learning. Reigeluth (1979) in his construction of his Elaboration theory viewed prerequisites, taught out of context, as too complex and lacking meaning.

Instead, his “epitome” serves as an advance organizer providing context and meaning as the smaller bits are examined. 10

Regardless of the organizational framework chosen by the designer, supports are typically incorporated into the instructional design. During instruction, these may take the form of advance organizers (Ausubel, 1960; Ausubel & Fitzgerald, 1961, 1962) providing frameworks (Chang, Sung, & Chen, 2002), chunking content to prevent cognitive overload (Sweller & Chandler, 1994), modeling (Collins, Brown, & Holum, 1991), or other learning aids (Ormrod, 2008). Collectively, these strategies are aimed at making learning accessible. As students transition to practice, the focus of this study, diﬀerent supports may be provided, e.g., coaching (Collins et al., 1991), tutoring or reciprocal teaching

(Chi, 1996; van den Boom, Paas, & van Merrienboer, 2007; El Saadawi et al., 2010;

Roll, Aleven, McLaren, & Koedinger, 2011) or providing feedback (Hattie, 2008). The goal is provide an approporiate amount support. Not so much as to render the practice meaningless, nor so little as to make it impossible for the learner to complete the session successfully. More precisely, the provided support should make the experience eﬀortful, but successful. The level of support is reduced as learner performance and skill increases, a process referred to as “fading”. The instructor moves to the side, their role diminished as the student assumes responsibility for the mastering the content.

Typically, practice aimed at improving performance quickly is characterized by little variability, strong supports, and considerable repetition. Students work through problems of a similar type before moving on to a diﬀerent class of problems. Due to its narrow focus and condensed nature, this form of practice is sometimes referred to as “massed practice” 11

(Kornell, Castel, Eich, & Bjork, 2010). Deliberate practice (Ericsson & Krampe, 1993) is a more sophisticated form of practice presented as a collection of strategies aimed at consistent improvement and applied to the pursuit of expertise. The theory of deliberate practice is derived from examinations of experts from a variety of professional domains, e.g., chess (Charness, Tuﬃash, Krampe, Reingold, & Vasyukova, 2005), music (Krampe

& Ericsson, 1996; Lehmann & Ericsson, 1997; Platz, Kopiez, Lehmann, & Wolf, 2014), sport (Baker, Cote, & Abernethy, 2003; Hodges, Kerr, Starkes, Weir, & Nananidou, 2004;

Baker, Cˆot´e,& Deakin, 2005; Ward, Hodges, Starkes, & Williams, 2007), and medicine

(Ericsson, 2004).

Deliberate practice preserves many of the strategies aimed at improving performance during or soon after acquisition, but recognizes the need to overcome an asymptotic plateau of performance associated with automaticity after a relatively short period of time practicing within a domain. This automaticity, or “loss of conscious control”, often appears after only 50 hours of practice. Once reached, continued work is done mindlessly resulting in no improvement in performance (Ericsson, 2008). The “key challenge for aspiring expert performers is to avoid the arrested development associated with automaticity” (p. 991).

Tasks for deliberate practice are to be just beyond what is able to be completed consistently at a high level by the individual, but are not to stretch the individual beyond what they would be able to master within the one-to-two hour time period typically 12 allotted (Ericsson, 2006). More broadly, deliberate practice requires a well-defined goal, motivation to improve, immediate (corrective) feedback, and ample opportunity to repeat tasks after making corrections in an effort to attain mastery (Ericsson, 2008). Deliberate practice is differentiated from play and performance in that the scope of activities is much smaller (Charness et al., 2005), and from traditional practice in that its singular focus on “stretch” results in a consistently onerous (uncomfortable) experience requiring great effort and motivation from the individual (Ericsson & Krampe, 1993).

Ericsson posits deliberate practice as virtually to sole differentiator between those who attain expertise, and those who do not. At its core, deliberate practice is a response to the age-old question of nature or nurture, one that discards innate (genetic) qualities and environmental factors in favor of prolonged, purposeful, motivated practice within a domain (Ericsson & Krampe, 1993). Exceptions are made only for the innate qualities of height and body size (Ericsson, 2014). This proposition has been met with strong opposition. For example, (Detterman & Ruthsatz, 1999) have proposed an alternative model, which accounts for general intelligence and domain-specific skills, as well as practice. Macnamara, Hambrick, and Oswald (2014) found deliberate practice to account for a much smaller, though still significant, amount of the variance in performances in the areas of games, sports, music, education, and professions. 1

1Notably, the authors found deliberate practice to account for only 1% of the variance for performance in professions, and only 4% of the variance for activities of “low predictability”, both of which might be used as descriptors for the nursing profession. 13

Ackerman (2014) notes Ericsson’s focus on experts explicitly eliminates those individuals who fail to possess the (assumingely) requisite innate qualities from his analyses.

Plomin, Shakeshaft, McMillan, and Trzaskowski (2014), using 10,000 12-year old twins found genetic factors account for more than half of the diﬀerence in performance between expert and normal readers. Chess mastery, at least at an early level, was found to be predicted by general intelligence (de Bruin, Kok, Leppink, & Camp, 2014), a contention supported more generally by literature review conducted by Grabner (2014). Hambrick et al. (2014) found only 34% of the variance in chess expertise could be attributed to deliberate practice. Similarly, the authors found deliberate practice to account for only 30% of the variance in music expertise. In an examination of child prodigies, Ruthsatz, Ruth- satz, and Stephens (2014) point out that a reanalysis of Ericsson’s data demonstrates those who eventually became elite musicians won more competitions at a very young age than those who eventually became only good or became music teachers, suggesting they might have possessed more innate talent initially. The authors also conclude that child prodigies abilities are at least partially based on innate talent. Finally, Wai (2014) retrospectively reviews the Study of Mathematically Precocious Youth, ﬁnding higher educational attainment by the top 1% and that increases in ability correlated with the percentage of degrees attained.

Ericsson has advocated for the application of deliberate practice to the medical field specifically (Ericsson, 2004; Ericsson et al., 2007; Ericsson, 2015). These efforts have 14 been successful as practitioners have successfully melded deliberate practice with simulations in areas such as surgical skills and cardiovascular examinations (Issenberg et al.,

1999), auscultation skills (Butter, McGaghie, Cohen, Kaye, & Wayne, 2010), study skills

(Duvivier et al., 2011), clinical skills Issenberg et al. (2002), advanced cardiac life support skills Wayne et al. (2006); Wayne, Didwania, et al. (2008), the improvement of thoracentesis skills Wayne, Barsuk, O’Leary, Fudala, and McGaghie (2008), various forms of catheter procedures Barsuk, Ahya, et al. (2009); Barsuk, McGaghie, Cohen, O’Leary, and

Wayne (2009); Barsuk, McGaghie, Cohen, Balachandran, and Wayne (2009), including the long-term retention of these skill (Barsuk, Cohen, McGaghie, & Wayne, 2010), overall academic achievement (Moulaert, Verwijnen, Rikers, & Scherpbier, 2004), and nursing students sense of clinical competence (Liou, Chang, Tsai, & Cheng, 2013a). However, as described in the introduction, the long-term retention of skills is suspect. In order to address the intermittent use of skills, some in the medical community have drawn from another area of research - desirable diﬃculties - in an attempt to improve long-term retention and transfer.

Improving Performance After a Delay

Strategies for improving long-term retention and transfer, collectively referred to as desirable difficulties (Bjork & Bjork, 1992), are based on manipulations of interference and time within practice sessions. These strategies make learning more difficult. The generation or testing effect and distributed practice are derived from manipulations of 15 time, while interleaving and contextual interference incorporate manipulations of both time and interference. Asking students to generate information is more powerful than simply asking them to recognize an answer amongst distractors, and has been demonstrated to improve performance on measures of long-term retention (Richland, Bjork,

Finley, & Linn, 2005), even in assisted situations such as open-book exams Agarwal et al. (2008). Repeating assessments even absent feedback can improve long-term retention

(Roediger & Karpicke, 2006b, 2006a). While generation itself has been shown to improve performance, these effects appear to be more significant when delays of some significance are placed between retrieval attempts (Rawson, Vaughn, & Carpenter, 2015).

Inserting time between generation attempts increases the eﬀort required for successful retrieval, a form of practice referred to as “spaced” or “distributed”. Spaced practice is often compared with massed practice, or repetitively studying an item or items on one focused, longer session. Distributed practice has a long history, rooted in the work of

Ebbinghaus, Bussenius, and Ruger (1913) in which he found that “with any considerable number of repetitions a suitable distribution of them over a space of time is decidedly more advantageous than the massing of them at a single time” (p. 89). Dempster (1988, p. 627) states that the spacing eﬀect is, “one of the most dependable and replicable phenomena in experimental psychology”. Studies have examined spaced practice as it relates to the recall of word pairs (Shaughnessy, 1977; Landauer & Bjork, 1978; Karpicke

& Roediger, 2007), mathematical knowledge Rohrer and Taylor (2007) and metacognition 16

Bahrick and Hall (2005). Various forms of spacing have been compared as well, with focuses on the length of the relationship between the inter-study and retention intervals

(Cepeda, Pashler, Vul, Wixted, & Rohrer, 2006; Cepeda, Vul, Rohrer, Wixted, & Pashler,

2008), as well as comparing equal versus expanding schedules of practice (Landauer &

Bjork, 1978; Karpicke & Roediger, 2007).

Rather than simply delay successive generation attempts, practitioners may interpolate other content or tasks within the practice schedule, creating a condition that incorporates both a delay and interference between attempts. This condition is called interleaved practice. And like spaced practice, it is often compared to massed or blocked practice. Eﬀorts have been made to determine the relative contributions of spacing and interference to the beneﬁts of interleaved practice (Taylor & Rohrer, 2010; Birnbaum,

Kornell, Bjork, & Bjork, 2012; Zulkiply & Burt, 2013). Generally, these studies have found the two contribute independently, with interference from interleaving contributing more signiﬁcantly. This research has produced a separate postulation related to category discriminability (Kang & Pashler, 2011), i.e., interleaved practice is beneﬁcial for high similarity categories as it facilitates between-category comparisons, while blocking is advantageous for low similarity categories as it increases within-category comparisons

(Kang & Pashler, 2011; Zulkiply & Burt, 2013; Carvalho & Goldstone, 2014b, 2014a;

Rohrer, Dedrick, & Burgess, 2014; Carvalho & Goldstone, 2015b, 2015a).

The beneﬁts of interleaved practice have been demonstrated for domains of art (Kornell 17

& Bjork, 2008; Kornell et al., 2010; Birnbaum et al., 2012; Zulkiply & Burt, 2013), mathematics (Rohrer & Taylor, 2007; Rau, Aleven, & Rummel, 2010, June; Taylor & Rohrer,

2010; Rau & Pardos, 2012; Rohrer, 2012; Li, Cohen, & Koedinger, 2013; Rau, Aleven,

& Rummel, 2013; Rohrer et al., 2014; Rohrer, Dedrick, & Stershic, 2015), astronomy

(Richland et al., 2005), chemistry (Li et al., 2013), and to a lesser extent, music (Carter

& Grahn, 2016). Notable results include Rohrer et al. (2014) who found interleaving to be nearly twice as effective as blocked practice, Rohrer et al. (2015) who found an effect size of 0.79 for interleaved mathematics learning after a delay of 30 days, and Kornell et al. (2010) who confirmed interleaved practice for inductive tasks to be superior even for older subjects. While interleaving most often involves alternating between two tasks, some studies interpolate three or more which is referred to as a “serial” condition (Shea &

Morgan, 1979; Goode & Magill, 1986). The ﬁnal manipulation, contextual interference, removes the predictable order of interleaved practice in favor of randomization.

Contextual interference most often refers to studies in which three or more tasks are practiced in random order, a strategy rooted in early work comparing random and logical sequencing. Randomized sequences of instruction were ﬁrst examined by Roe in two studies completed in 1962 (K. V. Roe, Case, & Roe, 1962; A. Roe, 1962). Subse- quent studies of college students (Payne, Krathwohl, & Gordon, 1967; Fry, 1972; Hiew,

1977; Van Merrienboer, De Croock, & Jelsma, 1997; de Croock, van Merrienboer, & Paas, 18

1998), high school students (Miller, 1969; Niedermeyer, Brown, & Sulzen, 1969), and children (Levin & Baker, 1963), incorporated topics ranging from mathematics (K. V. Roe et al., 1962; Levin & Baker, 1963; Payne et al., 1967; Miller, 1969; Niedermeyer et al.,

1969), to music (Hamilton, 1964), computer science (Fry, 1972), and poetry (Tennyson,

Steve, & Boutwell, 1975). The merits of blocked-random sequencing, in which items within logically-sequenced blocks of instruction were presented in random fashion (Levin

& Baker, 1963; Payne et al., 1967; Miller, 1969) have also been the focus of research.

Overall, this work is inconclusive, suggesting logical sequences of instruction increase performance during practice and the speed of acquisition, but fail to demonstrate consistent superiority in measures of short- or long-term retention and transfer. Researchers suggested the lack of “inter frame-conﬁrmation” (feedback) for random conditions forces learners to generate organizing principles (Hamilton, 1964; Payne et al., 1967), thus creating a more durable memory and improving long-term retention.

Battig’s (1972) work with word pairs, in which he initially coined the term “intratask interference”, marks the point at which interference and generally making things more difficult, e.g., removing supports such as feedback during the learning process, begins to be considered a theoretically valid instructional strategy. He concluded, “the facilitation of retention produced by interference within the acquisition task is of sufficient magnitude and generality to be of considerable theoretical as well as practical significance” (p.

155). In the ﬁrst experiment to demonstrate the contextual interference eﬀect in motor 19 skills, Shea and Morgan (1979) required subjects to knock down six freely moveable barriers in one of three prescribed patterns. Subjects in the control group performed

18 practice trials for each of the three patters in succession. Subjects in the blocked- random group performed blocks of 18 trials containing six of each of the three patterns randomly sequenced. Subjects in the control condition performed considerably faster during acquisition, but the time of the random group was comparable by the end of the acquisition phase. More importantly, the random group performed faster on all dependent measures, with the most signiﬁcant diﬀerence resulting from the transfer task.

In a follow up to this landmark study, Lee and Magill (1983) created a design to determine the origin of the superiority found for the random group in Shea and Mor- gan (1979). Four groups rather than two were employed: cued-blocked, cued-random, uncued-blocked, uncued-random. The authors found the cued condition to predict better performance during acquisition, but the random condition to be primarily responsible for improved retention. The contextual interference eﬀect has since been found in the practice of sports. Goode and Magill (1986) found a randomized schedule resulted in sig- niﬁcantly better performance on measures of retention and transfer of badminton serves.

Landin (1997) compared six different set basketball shot locations, finding the moder- ate CI group to perform significantly better on the post test when compared to the low and high CI groups, which did not differ. Hall, Domingues, and Cavazos (2017) found randomized schedules of practice to improve baseball players hitting ability by 56.7% 20 compared to 24.8% for the blocked condition and 6.2% for the control condition. In studies where the superiority of contextual interference has not been found, researchers have pointed to task similarity (Shewokis, 2003; Sugiyama, Araki, & Choshi, 2006) and ceiling effects (Meira & Tani, 2001) as explanations.

In addition to the very early work examining randomized sequences of instruction, variability of practice as it relates to cognitive skills has been examined in the areas of mathematics (F. G. W. C. Paas & van Merrienboer, 1994; F. Paas & van Gog, 2006), foreign language learning (Schneider, Healy, & Bourne Jr, 2002), and troubleshooting (van

Merrienboer, Kester, & Paas, 2006). In a series of experiments, a group of researchers with expertise in the field of cognitive load attempted to examine how contextual interference could be used to manipulate “germane cognitive load” in sequences aimed at teaching complex cognitive skills (Van Merrienboer et al., 1997; de Croock et al., 1998; de Croock & van Merriënboer, 2007). Both studies, the first exploratory in nature, asked subjects to troubleshoot an alcohol distillery plant. Four “case types” scenarios (valve malfunction, leakage, controller malfunction, and an alarm failure combined with a leak) were developed, each with 12 cases. Subjects in the blocked group received a set of problems of the same type, while subjects in the random group received a randomized sequence of problems from all types. The results suggest the randomized condition resulted in significantly better performance on measures of far transfer, and conclude by revisiting the transfer paradox, originally constructed in (Van Merrienboer et al., 1997), 21 which states that “high variability typically improves transfer performance for new variants of a task that were not practiced before, it may also impair performance during practice or require more training time to reach a pre-specified performance without pos- itive effects on performance at retention, that is, performance on variants of the task already practiced” (p. 784).

There are two primary competing explanations for the oft viewed superiority of these strategies. Shea and Morgan and Battig suggest that these eﬀects are due to elaboration during encoding, i.e., subjects integrating, comparing, and relating content to that already known or into existing schemata. This elaboration results in “richer representations” and “more distinctive memories” (Shea & Morgan, 1979; Shea & Zimny, 1983;

Lee & Simon, 2004). In contrast, Magill and Hall and Bjork believe that it is the act of forgetting a subsequently reconstructing the information that leads to enhanced retention and transfer (Lee & Magill, 1983, 1985; Bjork & Bjork, 1992; Meira & Tani, 2001; Lee &

Simon, 2004). Some eﬀorts have been made to disentangle the two primary theories. In one study, Young, Cohen, and Husak (1993) utilize a simple aiming task and interpolated activities to examine the two most oft proposed hypotheses, concluding that the two hypothesis are not mutually exclusive, and that they share two common characteristics that are essential for retention: eﬀortful processing and the facilitation of retrieval. In a more recent attempt, Lin, Fisher, Winstein, Wu, and Gordon (2008) applied Transcranial 22

Magnetic Simulation (TMS) during inter-trail intervals in an attempt to “directly per- turb hypothetical information processing”. The results of this study strongly supported the elaborative processing hypothesis.

Unlike deliberate practice, desirable diﬃculties have not been embraced by the medical community. Few examples can be found. Topics include electrocardiogram interpretation (interleaved practice, Hatala, Brooks, & Norman, 2003; Monteiro, Melvin,

Manolakos, Patel, & Norman, 2017), laparoscopic surgery techniques (spaced and interleaved practice, Goldin et al., 2014; Spruit, Band, & Hamming, 2015; Welsher &

Grierson, 2017), CPR (Patocka et al., 2015), and chest radiographies (interleaved practice, Rozenshtein, Pearson, Yan, Liu, & Toy, 2016). It does not appear that the tenets of contextual interference, speciﬁcally randomized or block-randomized schedules of practice, have been applied to the medical domain. Clapper (2011) may provide some insight as to why, arguing that contextual interference, randomized sequencing, and variability generally all serve to confuse the learner and therefore are to be avoided so that they can deliberately practice until they achieve mastery.

Summary

The reviewed research suggests strategies that optimizing performance during acquisition diﬀer from those that optimize performance after a delay. While the former strategies rely on strong supports, feedback, and repetition, the latter depends, to a large degree, on the absence of these qualities. Instead of strong guidance, learners are to 23 be consistently unsettled by the interpolating other content, randomizing of sequencing, or including delays between practice sessions. These “desirable diﬃculties” tend to depress performance during the acquisition phase of instruction, but often result in superior long-term retention and transfer.

Ericsson’s deliberate practice (1993) assumes many of the qualities required to optimize performance during acquisition, e.g., well-defined goals, motivation to improve, immediate (corrective) feedback, and ample opportunity to repeat tasks after making corrections in an effort to attain mastery (Ericsson, 2008). Additionally, deliberate practice requires consistent monitoring of progress and prompt modification of practice rou- tines to ensure learners are stretched during each session. Deliberate practice has been suggested as the path to expertise generally (Ericsson, 2006), and has been applied to the medical field specifically (Ericsson, 2004; Ericsson et al., 2007; Ericsson, 2015).

Oermann, Molloy, and Vaughn (2015) have advocated for deliberate practice serving as a core component for a redesign of medical training aimed at overcoming the inability of nascent medical students to perform procedures on actual patients. The medical commu- nities embrace of the patient simulators has allowed for many of Ericsson’s requirements to be met in programs preparing medical professionals of all kinds. Simulators paired with deliberate practice have shown to improve performance of nurses in the care of unstable and degrading patients (Whyte & Cormier, 2014), the bedside skills of medical students (Issenberg et al., 2002), and advanced cardiac life support skills (ALCS) 24

(Wayne et al., 2006). Moulaert et al. (2004) found the overall academic achievement of medical students correlated with the adoption of traits related to deliberate practice, e.g., self-study, study resources, planning and motivation.

However, the implementation of deliberate practice training programs is not straight- forward as it might seem. Broadly speaking, medical professionals begin training much later in life than those who served as subjects for Ericsson’s studies, e.g., world elites typically introduced to their ﬁeld of expertise between the ages of three and eight. This is signiﬁcant as attainment of expertise is said to require a minimum of 10,000 hours of deliberate practice. Only the medical specialist might approach this duration of formal training. Moreover, deliberate practice is inherently uncomfortable, requires timely feedback, consistent monitoring, and frequent adjustments to practice schedules to ensure appropriate stretch. These two requirements suggest only the most dedicated, intrin- sically motivated and self-regulated student might successfully advance from novice to expert.

On a more granular level, it is fair to question whether expertise is the wisest pursuit of a staff nurse charged with a myriad of tasks, some of which might only be executed on occasion. Nurses are expected, at times in stressful situations, to access previously learned but potentially latent skills with relative fluidity. A patient’s life may depend on their performance. An acceptable level of competence coupled with increased retrieval strength may be a more appropriate pursuit for these medical professionals, as opposed to 25 a surgeon who performs a limited number of highly technical procedures on a consistent basis. Introducing contextual interference, a desirable difficulty, might facilitate such an effect.

While desirable diﬃculties have been applied to the medical domain, this body of work is small relative to the work examining deliberate practice’s potential contributions. Much like deliberate practice, however, desirable diﬃculties research supports implementation for both cognitive and motor skills, requirements of the nursing profession. Deliberate practice strategies are already familiar, generally, to nurse educators.

Therefore, a study comparing deliberate practice with one made desirably difficult in an attempt to improve long-term retention is practical, as it would require only a small modification to practice schedule. The reviewed literature suggests block-randomization will depress performance during acquisition, result in comparable performance soon after training, and improved retention after a significant delay. The following hypotheses are proposed.

1. Do block-randomized and logical sequences of deliberately practiced nursing skill

procedures produce comparable performance on a post-assessment administered

soon after training?

(a) H0: procedural skills practiced in logical order will not result in superior per-

formance when compared to skills practiced in block-randomized order on a

post-assessment administered soon after training. 26

2. Do block-randomized and logical sequences of deliberately practiced nursing skill

procedures produce comparable performance on a delayed measure of long-term

retention?

(a) H0: there will be no signiﬁcant diﬀerence on a measure of long-term retention

for skills practiced in block-randomized and logical conditions 27

CHAPTER III

METHODS

The primary purpose of the proposed study was to determine whether block-randomization of deliberately-practiced nursing skills results in an increase in long-term retention. Nurse education literature presents deliberate practice and its alignment with the systematic pursuit of expertise as the preferred method for the practice of nursing skills. However, the performance of new nurses suggests skills learned during training are lost during periods of disuse. Strategies related to desirable diﬃculties have been shown to improve long-term retention. These strategies have been applied to medical education generally, but not to the practice of nursing skills speciﬁcally.

Participants

A total of 46 subjects began the study. However, absences on training days and attrition from second to third quarter of the program depressed participation. Thirty adult education LPN students at a small, midwestern vocational school, ranging in ages from 20 to 58 and averaging 30 years of age completed all components of the study. All possessed a high school diploma or GED. All but one subject required ﬁnancial assistance to enroll in the program. Thirteen of the 30 subjects had completed varying degrees of post-secondary coursework. Twenty-nine of the 30 subjects were female. Subjects were 28 not compensated for participating in the study as it represented only a small modiﬁcation to the sequence of required curriculum.

Materials

The primary resource used for selecting the nursing skills used in this study was

Fundamental Nursing Skills and Concepts (Timby, 2016). Editions of text have been used by instructors at the participating institution to guide the laboratory experiences for the past seven years. Two skills were chosen for practice: venipuncture and sterile dressing change. The former requires assessment of the client’s willingness to cooperate with the procedure, the identification fo an appropriate site for venipuncture, properly applying a tourniquet, palpating the vein, sterilizing the site, drawing the blood specimen, withdrawing the needle from the client, and attending to the puncture site. The principle steps of the latter include assessing the client’s pain level, confirming analgesic medication, properly positioning the patient, removing and assessing the old dressing, assessment and measurement of the wound, preparing sterile field, cleansing the wound with normal saline, drying the wound, re-taping the wound, and assessing patient comfort. The author and nursing manager generated a list of supplies required to run the requisite number of trials based on the class size. These supplies were ordered through the institution’s purchasing procedures.

ATI Testing’s Skills Modules served as the pre- and post-assessments for sterile dressing change and consisted of nine multiple-choice questions. Teacher-created assessments 29 of similar length (eight multiple-choice questions) were utilized for venipuncture, as ATI

Testing’s skills modules did not include assessments for this skill. Checklists, taken from

Timby (2016) and divided into three blocks of relatively equal length were utilized to assess subject performance during each practice session and skills assessment (see Appendix

A). Checklists also allowed for the assignment of percentage scores based on subject performance. Staﬀ and observers noted whether skill-steps were completed correctly, or incorrectly (as demonstrated by the need for the instructor to redirect/correct behavior).

Procedures

Nursing faculty conducted the training experiences. All of the instructors had previ- ous experience teaching skills in the lab or in the clinical setting. Staﬀ orientation took place one-week prior to the commencement of the study. The timeline, processes, and procedures were reviewed and a broad overview of the underlying rationale was provided.

Checklists were distributed and reviewed. Skill training took place over the course of two days, separated by one week. Training was completed in two nursing laboratories, one dedicated to procedural training, and the other dedicated to the block-random condition.

Nine staff members conducted training; three led the procedural training and six were responsible for the block-random training. Each skill was divided into three “skill blocks” (see Appendix A and B). Subjects were divided into two main groups. Group A practiced venipuncture procedurally during the first session, and sterile dressing change in blocked-random fashion during the second session. Practice conditions were flipped 30 for Group B (Table 1), i.e., the same skill was practiced during each session, with half the subjects practicing procedurally, and the other half practicing in block-random fashion.

Table 1: Within-Subjects Design

Main Group Session 1 Session 2

A Venipuncture Sterile Dressing Change

B Venipuncture Sterile Dressing Change

Note: Grayed cells represent the blocked-random condition.

Due to the number of subjects and skill blocks, requiring each subject to practice every skill block three times was impossible within the conﬁnes of the curricular scope and sequence of the program. Therefore, subjects within each group were divided into training subgroups of three. Training subgroups practiced each skill block three times, regardless of condition. Each member of the training group observed each block twice, and perform each skill block once. In the procedural condition, each member completed the skill blocks from beginning to end one after the other. In the block-random condition, each group moved from station to station in random fashion. Both the order of skill block (ﬁrst, second, or third) and the member of the group performing the skill block were predetermined, ensuring each subject practiced each block once and observed each block twice. This was done to ensure consistency between practice conditions. In both conditions, observers used the provided checklist to assess their training group member’s 31 progress as they attempted to perform each skill without the aid of the list. Figure 1 illustrates the two practice sequences, and Figure 2 summarizes the study’s design overall.

In accordance with the prescriptions for deliberate practice, each practice session began with the instructor formally stating the goal of the skill block. This was true in the procedural condition as well, i.e., although subjects completed blocks in order (one, two, three), they paused between blocks to ask them to state the objective. Subjects were then asked to begin practice. Corrections were made and feedback provided in real time. In order to ensure consistency, instructors provided feedback after each step - conﬁrmatory in instances in which the step was completed correctly, and corrective in instances where it was not. Instructors and observers noted corrections made throughout. At the conclusion of each practice session, subjects were asked to reﬂect on their performance by responding verbally to the prompt, “Please evaluate your performance”. Upon completion of this sequence, training groups in the random condition moved on to their next (randomly chosen) skill block station. Training groups in the logical condition completed all training at a single station, proceeding from one block to the next within the logical sequence.

Coding

Percent scores for each assessment were calculated for all three assessments. On the measure of long term retention, this was based on the number of correct skill-steps performed compared to the total number of skill steps. In order to assess diﬀerences in performance over time, subject performance was compared for each skill and condition 32

Figure 1: Example Study Sequences 33

Pre-Assessment Classroom Instruction (1) (2)

Laboratory Instruction Post-Assessment (3) (4)

21 Day Delay

Skills Assessment (5)

Figure 2: Study Design 34 prior to and immediately following practice, as well as after a delay 21 days. A repeated measures ANOVA was utilized with practice condition as the between-subject variable, time as the within subjects variable, and performance as the dependent variable. 35

CHAPTER IV

ANALYSIS OF THE FINDINGS

Two by three repeated measures ANOVAs were completed for both skills. Practice condition, procedural or block-random, served as the between subjects variable. Time was utilized as the within subjects variable. The interaction of time and practice condition was also analyzed.

Sterile Dressing Change

There was no between subject eﬀect for practice condition, F < 1. A main eﬀect was observed for time, the within subjects variable, F(2,56) = 192.95, MSe = 138.69,

2 p < .001, ηp = .31. No interaction between practice condition and time was realized,

F < 1. Within-subject contrasts demonstrated subject performance tracked closely and improved over time for both practice conditions as evidenced by Figure 3. 36

Figure 3: Subject performance: sterile dressing change 37

Venipuncture

The between subject eﬀect for practice condition approached signiﬁcance, F(1,28),

2 MSe = 58.95, p = .07, ηp = .11. The within subjects variable, time, proved to be

2 signiﬁcant, F(2,56) = 62.45, MSe = 162.74, p < .001, ηp = .69. The interaction between

2 practice condition and time was marginally signiﬁcant, F(2,56) = 2.60, p = .08, ηp = .09.

Notably, subject performance decreased on the post assessment for both practice conditions, but more dramatically for the block-random subjects. Within-subjects contrasts revealed a signiﬁcant diﬀerence in subject performance between measures two (the post

2 assessment) and three (the measure of long-term retention), F(1,28) = 5.24, p = .03, ηp

= .16. Subject performance over time is summarized in Figure 4.

Figure 4: Subject performance: venipuncture 38

CHAPTER V

DISCUSSION, LIMITATIONS, AND RECOMMENDATIONS

This study was designed to investigate the potential eﬀects of block-randomized practice on the long-term retention of nursing skills. Two research questions informed its design.

1. Do block-randomized and logical sequences of deliberately practiced nursing skill

procedures produce comparable performance on a post-assessment administered

soon after training?

2. Do block-randomized and logical sequences of deliberately practiced nursing skill

procedures produce comparable performance on a delayed measure of long-term

retention?

These questions served different purposes. The former seeks to confirm the generally observed finding that the two training schedules typically do not result in statistical differences in performance on measures of retention administered soon after training.

The latter examines whether the two schedules contribute to measure’s of long-term retention. This is the primary impetus of the study, as extant literature suggests both

(a) block-random schedules of practice may improve long-term retention, and (b) new 39 nurses struggle to retain satisfactory skill levels for skills comprising nursing preparation programs; skills those individuals demonstrated an acceptable level of proficiency performing during training. Moreover, modification of practice schedules from procedural to block-random would require little effort and virtually no cost to training institutions.

The results provide additional support to the idea that block-randomized schedules of practice result in comparable performance to traditional, procedural sequences on measures administered soon after training. These results are notable due to the nature of the topic. The majority of studies examining desirable difficulties have been focused on the areas of motor skills. Nursing care requires the application of both knowledge and skill, and is procedural in nature. Moreover, the small number of studies in the medical field rely on spaced or interleaved practice. To the author’s knowledge, this study represents the first attempt to support the contextual interference effect in the acquisition of procedural skills generally, and to the medical field specifically. However, the results do not support the potential superiority of the block-randomized schedule of practice of procedural skills on measures of long-term retention. Subjects performed comparably on this measure. While the null hypothesis was not rejected in this case, the comparable performance encourages further consideration as to the necessity of logical sequencing even for procedural skills. 40

Limitations

The primary limitations of this study are related to its integrated nature. While eﬀorts were made to control for variables inherent to a design utilizing an existing classroom cohort and course content, these eﬀorts were likely moderately successful at best. These limitations are enumerated below.

1. diﬀering levels of background knowledge

2. material diﬀerences in assessment formats

3. multiple physical locations

4. pre-announced measure of long-term retention

The composition of the cohort varied greatly, both in terms of age and background experience. Some subjects were relatively young having just graduated from high school, while others were significantly older and engaging in retraining with the aim of gaining better employment. The range of ages resulted in varying levels of background experience, specifically in the healthcare field. While none of the the subjects were previously nurses, some worked as nurse aides. Two subjects had previously completed venipuncture training and worked as certified phlebotomists. As it relates to this study, these individuals - to varying degrees - might have skewed the results on the assessments due to their extant, related background knowledge. 41

Due to time constraints, pre- and post-assessments were administered in multiple choice format, while training and the measure of long-term retention were based on performance. Questions were related either to background or procedural knowledge.

Same examples:

• “The recommended angle of insertion for a deep vain is degrees.”

• “A nurse is caring for a patient who has a heavily draining wound that continues

to show evidence of bleeding. Which of the following types of dressings should the

nurse select to help promote hemostasis?”

• “Wen cleaning the skin for a venipuncture procedure, the nurse uses the following

to cleanse the site. Select all that apply.”

• “A nurse is caring for a patient who has multiple sclerosis and has a chronic non-

healing wound. The nurse should recognize that which of the following types of

medications is known to delay wound healing?”

While the content was consistent and directly related, it would be unwise to equate background knowledge or knowing “what to do” with performing the skill itself. For example, a subject may know that 30o is the recommended angle for insertion into a deep vain, but lack the ﬁne motor skills required to execute such a procedure without more practice than was allotted in the study. Moreover, questions such as the second example address background knowledge that might, have been asked during the performance of 42 the skill, but that was not part of the formal assessment checklist. In summary, it is unclear to what degree the skill training that took place between the two assessments contributed to the decision-making processes and background knowledge underlying the performance of that skill.

Three physical locations were utilized for the study, a classroom and two labs. Stu- dents were randomly assigned to groups, and a master schedule was created to guide training and assessment sessions. The classroom served as a staging area, but was also where post-assessments were completed after returning from training in the labs. This area was monitored by the researcher, but the diﬀering durations of the procedural and block-random training conditions, and the diﬀerences between groups within each condition meant much time was spent organizing and revising scheduled start times throughout each sessions. This resulted in a less-than-ideal post-test environment and left little time for the researcher to monitor the subjects taking the post-test at any point during the session. An ad-hoc survey was created to ascertain to what degree subjects adhered to the stated directive to not utilize outside resources when completing the post-assessment.

While only ten subjects completed this survey, two of the respondents indicated they utilized online resources when completing the venipuncture post-assessment. Only one of the two subjects completed all components of the study. An analysis of the venipuncture data absent this subject’s contribution did not result in a change in the ﬁndings.

The most significant limitation of this study was the pre-announced nature of the 43 measure of long-term retention. The Ohio State Board of Nursing requires students in the program attain a passing score on the performance of the skills in this study. Due to the amount of time and staff resources required to assess all of the students on both skills utilizing the various formats, the measure of long-term retention also served as the assessment for the state board. Thus, students were made aware of the date and time of the skill assessment. This almost certainly skewed the results on the post-assessment due to significant investments of time preparing on the part of the students, as evidenced by high overall averages on the measure of long term retention for both sterile wound change (97.857%) and venipuncture (95.610%). This universally excellent performance on the measure of long-term retention is noteworthy and could be a reason why significant differences on the measure of long-term retention was not observed.

Lastly, a review of the checklists completed by the trainers reflected a potentially inconsistent application of the practice protocol. A significant percentage of the checkboxes for the goal statement and reflection, to be completed prior-to and following training, respectively, were left unchecked. This suggests these components of deliberate practice were not rigidly applied.

Recommendations

The pre-announced nature of the measure of long-term retention is a glaring weakness of this study. It obfuscates any insight into the primary purpose of the research, and calls for further investigation. Taken as a whole, this work indicates little if any penalty for 44 sequencing skill training in block-random fashion in the near-term, while leaving open the question of whether there might be some benefit over the long-term. A more tightly controlled study, utilizing applied pre- and post-assessments, in addition to an unannounced measure of long-term retention could answer these questions more definitively. A lab of programmable, high fidelity simulators would allow for the automation and precise control of such a study. APPENDIXES APPENDIX A

VENIPUNCTURE CHECKLISTS 47

Appendix A

Venipuncture Checklists

Venipuncture: Block One

Skill Correct Incorrect

Check for allergies.

Raise bed to good working height.

Assess client for possible risks associated with venipuncture.

Determine client’s ability to cooperate with procedure.

Assess client for contradicted sites for venipuncture; pres- ence of IV ﬂuids, hematoma at potential site, arm on side of mastectomy or hemodialysis shunt.

Assist client to supine or semi-Fowler’s position.

Apply tourniquet 2 – 4 inches above intended site (ante- cubital fossa is most often used) - keep on no longer than

1 – 2 minutes. 48

Venipuncture: Block One (continued)

Skill Correct Incorrect

Ask client to open and close fists several times, finally leaving fists clenched.

Quickly inspect extremity for best venipuncture site, looking for straight, prominent vein without swelling or hematoma.

Palpate vein with pads of ﬁngers – note if vein feels elas- tic, bouncy, and rebounds when palpated. Feel for the shape and direction of the vein along with the size. Avoid scars, lesions, bruising, edema 49

Venipuncture: Block Two

Skill Correct Incorrect

Select venipuncture site.

Attach needle to vacutainer tube

Have proper blood specimen tube resting inside vacutainer, but do not puncture rubber stopper.

Cleanse site with alcohol swab or appropriate method of bacteriostatic solution moving in a circular motion out from intended puncture site approximately 2 inches.

Allow the site to dry for 30 seconds

Remove needle cover and inform patient of “stick” lasting only a few seconds will be felt.

Place thumb or foreﬁnger of non-dominant hand 1 inch below site and pull skin taut. Stretch skin down until vein stabilized. 50

Venipuncture: Block Two (continued)

Skill Correct Incorrect

Hold vacutainer needle at 15-30-degree angle from arm with bevel up- steeper angle increases risk of passing through vein into deeper structures.

Slowly insert into vein in a swift continuous motion (monitor for a sight “popping sensation or release of resistance against the needle point— which you will feel when you enter the vein, and note flow of blood; with butterfly you will see an immediate flashback).

Grasp vacutainer securely and advance specimen tube into needle of holder.

Note ﬂow of blood into tube (should be fairly rapid). 51

Venipuncture: Block Three

Skill Correct Incorrect

After specimen tube is ﬁlled, grasp vacutainer ﬁrmly with non-dominant hand and remove tube.

While the last tube is ﬁlling, remove the tourniquet by gently tugging at the tucked end

After last tube ﬁlled, remove from vacutainer holder and withdraw the needle from the client’s arm

Immediately apply 2x2 gauze pad to hold pressure on puncture site for 1-2 minutes – it is best to refrain from

“peeking” or lifting gauze until at least 1 min has passed.

Activate the needle safety device promptly

Do not allow the patient to ﬂex the elbow to hold pressure on the site- it tends to increase hematoma formation.

Tape gauze securely over site.

Instruct to remove after 1-2 hours. APPENDIX B

STERILE DRESSING CHANGE CHECKLISTS 53

Appendix B

Sterile Dressing Change Checklists

Sterile Dressing Change: Block One

Skill Correct Incorrect

Assess pain level.

Verbalize that client would have been medicated with an analgesic 30 minutes prior to painful wound care.

Riase bed to working height.

Position patient correctly to expose wound only.

Place absorent pad under wound.

Remove old dressing by loosening edge of the tap stabi- lizing the skin with one hand while pulling the tape in the opposite direction toward the wound.

Assess old dressing and wound; discards old dressing and gloves in biohazard bag.

Perform hand hygiene.

Measure length of wound in centimeters. 54

Sterile Dressing Change: Block Two

Skill Correct Incorrect

Sets up sterile field considering principles of asepsis: any- thing below waist is unsterile; sterile field always in field of vision.

Properly open sterile towel with sterile bowl.

Properly add normal saline to bowl (palming label, pours oﬀ, does not reach across sterile ﬁeld).

Open and place two 4x4 gauzes and ABD correctly onto sterile ﬁeld.

Don sterile gloves.

Aseptically cleanse wound using normal saline and one 4 x 4 gauzes, (one swipe in one direction with each stroke/ clean to dirty).

Use the second 4 x 4 gauze to pat dry wound from top to bottom. 55

Sterile Dressing Change: Block Three

Skill Correct Incorrect

Pick up and apply dry sterile ABD dressing to wound

(avoids unnecessary touching of underside of dressing).

Neatly (picture frame) tape wound.

Place piece of tape with initials, date, and time on dressing.

Return client to safe position.

Assess client for comfort.

Give client call light.

Remove all soiled supplies.

Discard gloves.

Perform hand hygiene.

Record and document procedure. Including (in this order): Date & Time, Removal of and Assessment of old dressing and wound; Wound Cleansing with Normal

Saline (NS), Application of Dry Sterile Dressing and Se- curing of dressing with tape; Client Response to Proce- dure, and signed by SPN. REFERENCES REFERENCES

Ackerman, P. L. (2014). Nonsense, common sense, and science of expert performance:

Talent and individual diﬀerences. Intelligence, 45 , 6–17.

Agarwal, P. K., Karpicke, J. D., Kang, S. H., Roediger, H. L., & McDermott, K. B.

(2008). Examining the testing eﬀect with open-and closed-book tests. Applied

Cognitive Psychology, 22 (7), 861–876.

Ahlberg, G., Enochsson, L., Gallagher, A. G., Hedman, L., Hogman, C., McClusky,

D. A., . . . Arvidsson, D. (2007). Proﬁciency-based virtual reality training sig-

niﬁcantly reduces the error rate for residents during their ﬁrst 10 laparoscopic

cholecystectomies. The American Journal of Surgery, 193 (6), 797–804.

Andreatta, P. B., Woodrum, D. T., Birkmeyer, J. D., Yellamanchilli, R. K., Doherty,

G. M., Gauger, P. G., & Minter, R. M. (2006, June). Laparoscopic Skills Are

Improved With LapMentor™ Training: Results of a Randomized, Double-Blinded

Study. Annals of Surgery, 243 (6), 854–863.

Arthur Jr., W., Bennett Jr., W., Stanush, P. L., & McNelly, T. L. (1998, 03). Factors

that inﬂuence skill decay and retention: A quantitative review and analysis. Human

Performance, 11 (1), 57–101.

Ausubel, D. P. (1960). The use of advance organizers in the learning and retention of

meaningful verbal material. Journal of Educational Psychology, 51 (5), 267–272. 57 58

Ausubel, D. P., & Fitzgerald, D. (1961). The role of discriminability in meaningful

learning and retention. Journal of Educational Psychology, 52 (5), 266–274.

Ausubel, D. P., & Fitzgerald, D. (1962). Organizer, general background, and antecedent

learning variables in sequential verbal learning. Journal of Educational Psychology,

53 (6), 243–249.

Bahrick, H. P., & Hall, L. K. (2005). The importance of retrieval failures to long-term

retention: A metacognitive explanation of the spacing eﬀect. Journal of Memory

and Language, 52 (4), 566–577.

Baker, J., Cote, J., & Abernethy, B. (2003, January). Sport-Speciﬁc Practice and the

Development of Expert Decision-Making in Team Ball Sports. Journal of Applied

Sport Psychology, 15 (1), 12–25.

Baker, J., Cˆot´e,J., & Deakin, J. (2005, March). Expertise in Ultra-Endurance Triath-

letes Early Sport Involvement, Training Structure, and the Theory of Deliberate

Practice. Journal of Applied Sport Psychology, 17 (1), 64–78.

Barsuk, J. H., Ahya, S. N., Cohen, E. R., McGaghie, W. C., & Wayne, D. B. (2009).

Mastery Learning of Temporary Hemodialysis Catheter Insertion by Nephrology

Fellows Using Simulation Technology and Deliberate Practice. American Journal

of Kidney Diseases, 54 (1), 70–76.

Barsuk, J. H., Cohen, E. R., McGaghie, W. C., & Wayne, D. B. (2010). Long-term

retention of central venous catheter insertion skills after simulation-based mastery 59

learning. Academic Medicine, 85 (10), S9–S12.

Barsuk, J. H., McGaghie, W. C., Cohen, E. R., Balachandran, J. S., & Wayne, D. B.

(2009, September). Use of simulation-based mastery learning to improve the quality

of central venous catheter placement in a medical intensive care unit. Journal of

Hospital Medicine, 4 (7), 397–403.

Barsuk, J. H., McGaghie, W. C., Cohen, E. R., O’Leary, K. J., & Wayne, D. B. (2009).

Simulation-based mastery learning reduces complications during central venous

catheter insertion in a medical intensive care unit. Critical Care Medicine, 37 (10),

2697–2701.

Battig, W. F. (1972). Intratask Interference as a Source of Facilitation in Transfer and

Retention. In Topics in learning and performance.

Battig, W. F. (1979). The ﬂexibility of human memory. Levels of processing and human

memory, Lawrence Erlbaum Associates, Hillsdale, NJ , 23–44.

Berkow, S., Virkstis, K., Stewart, J., & Conway, L. (2008). Assessing New Graduate

Nurse Performance. Journal of Nursing Administration, 38 (11), 468–474.

Birnbaum, M. S., Kornell, N., Bjork, E. L., & Bjork, R. A. (2012). Why interleaving

enhances inductive learning: The roles of discrimination and retrieval. Memory &

Cognition, 41 (3), 392–402.

Bjork, E. L., & Bjork, R. A. (2009). Making things harder on yourself, but in a good

way: Creating desirable diﬃculties to enhance learning. In Psychology and the real 60

world: Essays illustrating fundamental contributions to society.

Bjørk, I. T., & Kirkevold, M. (1999). Issues in Nurses’ Practical Skill Development in

the Clinical Setting. Journal of Nursing Care Quality, 14 (1), 72–84.

Bjork, R. A. (1994). Memory and Metamemory Considerations in the Training of Human

Beings. In J. Metcalfe & A. Shimanura (Eds.), Metacognition: Knowing about

knowing (pp. 185–205).

Bjork, R. A., & Bjork, E. L. (1992). A New Theory of Disuse and an Old Theory of

Stimulus Fluctuation. In S. K. A. Healy & R. Shiﬀrin (Eds.), Learning processes

to cognitive processes: Essays in honor of william k. estes (pp. 35–67).

Bogossian, F., Cooper, S., Cant, R., Beauchamp, A., Porter, J., Kain, V., . . . Phillips,

N. M. (2014). Undergraduate nursing students’ performance in recognising and

responding to sudden patient deterioration in high psychological ﬁdelity simulated

environments: An Australian multi-centre study. Nurse Education Today, 34 (5),

691–696.

Butter, J., McGaghie, W. C., Cohen, E. R., Kaye, M., & Wayne, D. B. (2010, Au-

gust). Simulation-based Mastery Learning Improves Cardiac Auscultation Skills in

Medical Students. Journal of General Internal Medicine, 25 (8), 780–785.

Carlisle, C., Luker, K. A., Davies, C., Stilwell, J., & Wilson, R. (1999, May). Skills compe-

tency in nurse education: nurse managers’ perceptions of diploma level preparation.

Journal of Advanced Nursing, 29 (5), 1256–1264. 61

Carter, C. E., & Grahn, J. A. (2016). Optimizing Music Learning: Exploring How

Blocked and Interleaved Practice Schedules Aﬀect Advanced Performance. Fron-

tiers in Psychology, 7 (1251), 1–10.

Carvalho, P. F., & Goldstone, R. L. (2014a). Eﬀects of interleaved and blocked

study on delayed test of category learning generalization. Frontiers in Psychol-

ogy, 5 (Agarwal), 1–11.

Carvalho, P. F., & Goldstone, R. L. (2014b). Putting category learning in order: Category

structure and temporal arrangement aﬀect the beneﬁt of interleaved over blocked

study. Memory & Cognition, 42 (3), 481–495.

Carvalho, P. F., & Goldstone, R. L. (2015a). The beneﬁts of interleaved and blocked

study: Different tasks benefit from different schedules of study. Psychonomic Bul-

letin & Review, 22 (1), 281–288.

Carvalho, P. F., & Goldstone, R. L. (2015b). What you learn is more than what you see:

what can sequencing eﬀects tell us about inductive category learning? Frontiers in

Psychology, 6 (505), 1–12.

Casey, K., Fink, R. R., Krugman, A. M., & Propst, F. J. (2004). The Graduate Nurse

Experience. Journal of Nursing Administration, 34 (6), 303–311.

Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed

practice in verbal recall tasks: A review and quantitative synthesis. Psychological

Bulletin, 132 (3), 354–380. 62

Cepeda, N. J., Vul, E., Rohrer, D., Wixted, J. T., & Pashler, H. (2008, November).

Spacing Eﬀects in Learning. Psychological science, 19 (11), 1095–1102.

Chang, K.-E., Sung, Y.-T., & Chen, I.-D. (2002). The eﬀect of concept mapping to en-

hance text comprehension and summarization. Journal of Experimental Education,

71 (1), 5–23.

Charness, N., Tuﬃash, M., Krampe, R., Reingold, E., & Vasyukova, E. (2005). The

role of deliberate practice in chess expertise. Applied Cognitive Psychology, 19 (2),

151–165.

Chi, M. T. H. (1996). Constructing self-explanations and scaﬀolded explanations in

tutoring. Applied Cognitive Psychology, 10 (Special), 33–49.

Clapper, T. C. (2011). Interference in Learning: What Curriculum Developers Need to

Know. Clinical Simulation in Nursing, 7 (3), e77–e80.

Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making things

visible. American Educator: The Professional Journal of the American Federation

of Teachers, 15 (3), 6–11,38–46. de Bruin, A. B. H., Kok, E. M., Leppink, J., & Camp, G. (2014). Practice, intelligence,

and enjoyment in novice chess players: A prospective study at the earliest stage of

a chess career. Intelligence, 45 , 18–25. de Croock, M. B. M., & van Merri¨enboer, J. (2007). Paradoxical eﬀects of information

presentation formats and contextual interference on transfer of a complex cognitive 63

skill. Computers in Human Behavior, 23 (4), 1740–1761. de Croock, M. B. M., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998). High versus

low contextual interference in simulation-based training of troubleshooting skills:

eﬀects on transfer performance and invested mental eﬀort. Computers in Human

Behavior, 14 (2), 249–267.

Dempster, F. N. (1988). The spacing eﬀect: A case study in the failure to apply the

results of psychological research. American Psychologist, 43 (8), 627–634.

Detterman, D. K., & Ruthsatz, J. (1999). Toward a more comprehensive theory of

exceptional abilities. Journal for the Education of the Gifted, 22 (2), 148–158.

Duvivier, R. J., van Dalen, J., Muijtjens, A. M., Moulaert, V. R., van der Vleuten, C. P.,

& Scherpbier, A. J. (2011). The role of deliberate practice in the acquisition of

clinical skills. BMC Medical Education, 11 (107), 1–7.

Ebbinghaus, H., Bussenius, C. E., & Ruger, H. A. (1913). Memory; a contribution to

experimental psychology. New York: Teachers College, Columbia University.

El Saadawi, G., Azevedo, R., Castine, M., Payne, V., Medvedeva, O., Tseytlin, E.,

. . . Crowley, R. (2010, March). Factors aﬀecting feeling-of-knowing in a medical

intelligent tutoring system: The role of immediate feedback as a metacognitive

scaﬀold. Advances in Health Sciences Education, 15 (1), 9–30.

Ericsson, K. A. (2004). Deliberate Practice and the Acquisition and Maintenance of Ex-

pert Performance in Medicine and Related Domains. Academic Medicine, 79 (10), 64

S70 – S81.

Ericsson, K. A. (2006). The Inﬂuence of Experience and Deliberate Practice on the De-

velopment of Superior Expert Performance. In The cambridge handbook of expertise

and expert performance. Cambridge University Press.

Ericsson, K. A. (2008, November). Deliberate practice and acquisition of expert perfor-

mance: a general overview. Academic emergency medicine : oﬃcial journal of the

Society for Academic Emergency Medicine, 15 (11), 988–994.

Ericsson, K. A. (2014). Why expert performance is special and cannot be extrapolated

from studies of performance in the general population: A response to criticism.

Intelligence, 45 , 81–103.

Ericsson, K. A. (2015). Acquisition and Maintenance of Medical Expertise: A Perspec-

tive From the Expert-Performance Approach With Deliberate Practice. Academic

Medicine, 90 (11), 1471–1486.

Ericsson, K. A., & Krampe, R. T. (1993). The role of deliberate practice in the acquisition

of expert performance. Psychological Review, 100 (3), 363–486.

Ericsson, K. A., Whyte, J. I., & Ward, P. (2007). Expert Performance in Nursing:

Reviewing Research on Expertise in Nursing Within the Framework of the Expert-

Performance Approach. Advances in Nursing Science, 30 (1), E58–E71.

Eva, K. W. (2005, January). What every teacher needs to know about clinical reasoning.

Medical Education, 39 (1), 98–106. 65

Fawcett, J. (1992, February). Conceptual models and nursing practice: the reciprocal

relationship. Journal of Advanced Nursing, 17 (2), 224–228.

Fry, J. P. (1972). Interactive relationship between inquisitiveness and student control of

instruction. Journal of Educational Psychology, 5 (459–465).

Goldin, S. B., Horn, G. T., Schnaus, M. J., Grichanik, M., Ducey, A. J., Nofsinger, C.,

. . . Brannick, M. T. (2014). FLS Skill Acquisition: A Comparison of Blocked vs

Interleaved Practice. Journal of Surgical Education, 71 (4), 506–512.

Goode, S., & Magill, R. A. (1986, December). Contextual Interference Eﬀects in Learning

Three Badminton Serves. Research Quarterly for Exercise & Sport, 57 (4), 308–314.

Grabner, R. H. (2014). The role of intelligence for performance in the prototypical

expertise domain of chess. Intelligence, 45 , 26–33.

Haji, F. A., Cheung, J. J. H., Woods, N., Regehr, G., Ribaupierre, S., & Dubrowski, A.

(2016, September). Thrive or overload? The eﬀect of task complexity on novices’

simulation-based learning. Medical Education, 50 (9), 955–968.

Hall, K. G., Domingues, D. A., & Cavazos, R. (2017, May). Contextual Interference

Eﬀects with Skilled Baseball Players . Perceptual & Motor Skills, 78 (3), 835–841.

Hambrick, D. Z., Oswald, F. L., Altmann, E. M., Meinz, E. J., Gobet, F., & Campitelli,

G. (2014). Deliberate practice: Is that all it takes to become an expert? Intelli-

gence, 45 , 34–45.

Hamilton, N. R. (1964). Eﬀects of logical versus random sequencing of items in an 66

autoinstructional program under two conditions of covert response. Journal of

Educational Psychology, 55 (5), 258–266.

Hatala, R. M., Brooks, L. R., & Norman, G. R. (2003). Practice Makes Perfect: The

Critical Role of Mixed Practice in the Acquisition of ECG Interpretation Skills.

Advances in Health Sciences Education, 8 (1), 17–26.

Hattie, J. (2008). Visible Learning. London and New York: Routledge.

Hickey, M. T. (2009). Preceptor Perceptions of New Graduate Nurse Readiness for

Practice. Journal for Nurses in Professional Development, 25 (1), 35–41.

Hiew, C. C. (1977). Sequence Eﬀects in Rule Learning and Conceptual Generalization.

The American Journal of Psychology, 90 (2), 207–218.

Hodges, N. J., Kerr, T., Starkes, J. L., Weir, P. L., & Nananidou, A. (2004). Predicting

Performance Times From Deliberate Practice Hours for Triathletes and Swimmers:

What, When, and Where Is Practice Important? Journal of Experimental Psy-

chology: Applied, 10 (4), 219–237.

Issenberg, S. B., McGaghie, W. C., Gordon, D. L., Symes, S., Petrusa, E. R., Hart, I. R.,

& Harden, R. M. (2002, October). Eﬀectiveness of a Cardiology Review Course for

Internal Medicine Residents Using Simulation Technology and Deliberate Practice.

Teaching and Learning in Medicine, 14 (4), 223–228.

Issenberg, S. B., McGaghie, W. C., & Hart, I. R. (1999, September). Simulation Technol-

ogy for Health Care Professional Skills Training and Assessment. JAMA, 282 (9), 67

861–866.

Issenberg, S. B., McGaghie, W. C., Petrusa, E. R., Lee Gordon, D., & Scalese, R. J.

(2009, July). Features and uses of high-ﬁdelity medical simulations that lead to

eﬀective learning: a BEME systematic review. Medical Teacher, 27 (1), 10–28.

Jeﬀries, P. R. (2005, March). A framework for designing, implementing, and evaluating

simulations used as teaching strategies in nursing. Nursing education perspectives,

26 (2), 96–103.

Kaakinen, J., & Ellyn, A. (2009). Systematic Review of Nursing Simulation Literature for

Use of Learning Theory. International Journal of Nursing Education Scholarship,

6 (1), 1–20.

Kahneman, D. (2011). Thinking, fast and slow. Allen Lane, UK.

Kang, S. H. K., & Pashler, H. (2011, May). Learning Painting Styles: Spacing is Advan-

tageous when it Promotes Discriminative Contrast. Applied Cognitive Psychology,

26 (1), 97–103.

Karpicke, J. D., & Roediger, H. L. (2007). Expanding retrieval practice promotes short-

term retention, but equally spaced retrieval enhances long-term retention. Journal

of Experimental Psychology / Learning, Memory & Cognition, 33 (4), 704–719.

Korndorﬀer, J. R., Dunne, J. B., Sierra, R., Stefanidis, D., Touchard, C. L., & Scott,

D. J. (2005). Simulator Training for Laparoscopic Suturing Using Performance

Goals Translates to the Operating Room. Journal of the American College of 68

Surgeons, 201 (1), 23–29.

Korndorﬀer, J. R., Hayes, D. J., Dunne, J. B., Sierra, R., Touchard, C. L., Markert,

R. J., & Scott, D. J. (2005). Development and transferability of a cost-eﬀective

laparoscopic camera navigation simulator. Surgical Endoscopy, 19 (2), 161–167.

Kornell, N., & Bjork, R. A. (2008). Learning concepts and categories: is spacing the

”enemy of induction”? Psychological science, 19 (6), 585–592.

Kornell, N., Castel, A. D., Eich, T. S., & Bjork, R. A. (2010). Spacing as the friend

of both memory and induction in young and older adults. Psychology and Aging,

25 (2), 498–503.

Krampe, R. T., & Ericsson, K. A. (1996). Maintaining excellence: Deliberate practice and

elite performance in young and older pianists. Journal of Experimental Psychology:

General, 125 (4), 331–359.

Landauer, T. K., & Bjork, R. A. (1978). Optimal rehearsal patterns and name learning.

In Practical aspects of memory. Academic Press: London.

Landin, D. (1997). A Comparison of Three Practice Schedules along the Contextual

Interference Continuum. Research Quarterly for Exercise & Sport, 68 (4), 357–

361.

Lee, T. D., & Magill, R. A. (1983). The locus of contextual interference in motor-skill

acquisition. Journal of Experimental Psychology / Learning, Memory & Cognition,

9 (4), 730–746. 69

Lee, T. D., & Magill, R. A. (1985). Can Forgetting Facilitate Skill Acquisition? Advances

in Psychology, 27 , 3–22.

Lee, T. D., & Simon, D. A. (2004). Contextual Interference. In M. A. Williams &

N. J. Hodges (Eds.), Skill acquisition in sport: Research, theory and practice (pp.

29–44).

Lehmann, A. C., & Ericsson, K. A. (1997). Research on expert performance and de-

liberate practice: Implications for the education of amateur musicians and music

students. Psychomusicology: A Journal of Research in Music Cognition, 16 (1-2),

40–58.

Levin, G. R., & Baker, B. L. (1963). Item scrambling in a self-instructional program.

Journal of Educational Psychology, 54 (3), 138–143.

Li, N., Cohen, W. W., & Koedinger, K. R. (2013, October). Problem Order Implications

for Learning. International Journal of Artiﬁcial Intelligence in Education, 23 (1-4),

71–93.

Lin, C.-H. J., Fisher, B. E., Winstein, C. J., Wu, A. D., & Gordon, J. (2008). Contex-

tual Interference Eﬀect: Elaborative Processing or Forgetting—Reconstruction? A

Post Hoc Analysis of Transcranial Magnetic Stimulation—Induced Eﬀects on Motor

Learning. Journal of Motor Behavior, 40 (6), 578–586.

Liou, S. R., Chang, C. H., Tsai, H. M., & Cheng, C. Y. (2013a). The eﬀects of a

deliberate practice program on nursing students’ perception of clinical competenc. 70

Nurse Education Today, 33 (4), 358–363.

Liou, S.-R., Chang, C.-H., Tsai, H.-M., & Cheng, C.-Y. (2013b). The eﬀects of a

deliberate practice program on nursing students’ perception of clinical competence.

Nurse Education Today, 33 (4), 358–363.

Macnamara, B. N., Hambrick, D. Z., & Oswald, F. L. (2014, June). Deliberate Practice

and Performance in Music, Games, Sports, Education, and Professions. Psycholog-

ical science, 25 (8), 1608–1618.

Mager, R. F. (1961). On the sequencing of instructional content. Perceptual & Motor

Skills, 9 (2), 405–413.

Magill, R. A., & Hall, K. G. (1990). A review of the contextual interference eﬀect in

motor skill acquisition. Human Movement Science, 9 (3–5), 241–289.

McGaghie, W. C., Issenberg, S. B., Cohen, E. R., Barsuk, J. H., & Wayne, D. B. (2011,

June). Does Simulation-based Medical Education with Deliberate Practice Yield

Better Results than Traditional Clinical Education? A Meta-Analytic Compara-

tive Review of the Evidence. Academic medicine : journal of the Association of

American Medical Colleges, 86 (6), 706–711.

McGaghie, W. C., Issenberg, S. B., Petrusa, E. R., & Scalese, R. J. (2006, August).

Eﬀect of practice on standardised learning outcomes in simulation-based medical

education. Medical Education, 40 (8), 792–797.

Meira, C. M., & Tani, G. O. (2001). The Contextual Interference Eﬀect in Acquisition of 71

Dart-Throwing Skill Tested on a Transfer Test With Extended Trials. Perceptual

& Motor Skills, 92 (3), 910–918.

Miller, H. R. (1969). Sequencing and prior information in linear programed instruction.

AV Communication Review, 17 (1), 63–76.

Monteiro, S., Melvin, L., Manolakos, J., Patel, A., & Norman, G. (2017). Evaluating the

eﬀect of instruction and practice schedule on the acquisition of ECG interpretation

skills. Perspectives on Medical Education, 6 (4), 237–245.

Moulaert, V., Verwijnen, M. G. M., Rikers, R., & Scherpbier, A. J. J. A. (2004, October).

The eﬀects of deliberate practice in undergraduate medical education. Medical

Education, 38 (10), 1044–1052.

Nehring, W. M., & Lashley, F. R. (2008, December). Nursing Simulation: A Review of

the Past 40 Years. Simulation & Gaming, 40 (4), 528–552.

Niedermeyer, F., Brown, J., & Sulzen, B. (1969). Learning and varying sequences

of ninth-grade mathematics materials. The Journal of Experimental Educational,

16 (3), 301–317.

Oermann, M. H., Molloy, M. A., & Vaughn, J. (2015, April). Use of deliberate practice

in teaching in nursing. Nurse Education Today, 35 (4), 535–536.

Ormrod, J. E. (2008). Human Learning. Upper Saddle River, New Jersey: Pearson.

Paas, F., & van Gog, T. (2006). Optimising worked example instruction: Diﬀerent ways

to increase germane cognitive load. Learning & Instruction, 16 (2), 87–91. 72

Paas, F. G. W. C., & van Merrienboer, J. J. G. (1994). Variability of worked examples and

transfer of geometrical problem-solving skills: A cognitive-load approach. Journal

of Educational Psychology, 86 (1), 122–133.

Patocka, C., Khan, F., Dubrovsky, A. S., Brody, D., Bank, I., & Bhanji, F. (2015).

Pediatric resuscitation training—Instruction all at once or spaced over time? Re-

suscitation, 88 (Supplement C), 6–11.

Payne, D. A., Krathwohl, D. R., & Gordon, J. (1967). The Eﬀect of Sequence on

Programmed Instruction. American Educational Research Journal, 4 (2), 125–132.

Platz, F., Kopiez, R., Lehmann, A. C., & Wolf, A. (2014, June). The inﬂuence of delib-

erate practice on musical achievement: a meta-analysis. Frontiers in Psychology,

5 , 646.

Plomin, R., Shakeshaft, N. G., McMillan, A., & Trzaskowski, M. (2014). Nature, nurture,

and expertise. Intelligence, 45 , 46–59.

Posner, G. J., & Strike, K. A. (1976). A Categorization Scheme for Principles of Se-

quencing Content. Review of Educational Research, 46 (4), 665–690.

Rau, M. A., Aleven, V., & Rummel, N. (2010, June). Blocked versus Interleaved Practice

with Multiple Representations in an Intelligent Tutoring System for Fractions. In

V. Aleven, J. Kay, & J. Mostow (Eds.), Intelligent tutoring systems (pp. 413–422).

Berlin, Heidelberg: Springer, Berlin, Heidelberg.

Rau, M. A., Aleven, V., & Rummel, N. (2013). Interleaved practice in multi-dimensional 73

learning tasks: Which dimension should we interleave. Learning & Instruction, 23 ,

98–114.

Rau, M. A., & Pardos, Z. A. (2012, June). Interleaved Practice with Multiple Repre-

sentations: Analyses with Knowledge Tracing Based Techniques (Tech. Rep. No.

ED537218). Chania, Greece: Carnegie Mellon University.

Rawson, K. A., Vaughn, K. E., & Carpenter, S. K. (2015). Does the beneﬁt of testing

depend on lag, and if so, why? Evaluating the elaborative retrieval hypothesis.

Memory & Cognition, 43 (4), 619–633.

Reigeluth, C. M. (1979). In Search of a Better Way to Organize Instruction: The

Elaboration Theory. Journal of Instructional Development, 2 (3), 8–15.

Richland, L. E., Bjork, R. A., Finley, J. R., & Linn, M. C. (2005). Linking cognitive

science to education: Generation and interleaving eﬀects. In Proceedings of the

twenty-seventh annual conference of the cognitive science society (pp. 1850–1855).

Roe, A. (1962). A Comparison of Branching Methods for Programmed Learning. The

Journal of Educational Research, 55 (9), 407–416.

Roe, K. V., Case, H. W., & Roe, A. (1962). Scrambled versus ordered sequence in

autoinstructional programs. Journal of Educational Psychology, 53 (2), 101–104.

Roediger, H. L., & Karpicke, J. D. (2006a). The Power of Testing Memory: Basic

Research and Implications for Educational Practice. Perspectives on Psychological

Science, 1 (3), 181–210. 74

Roediger, H. L., & Karpicke, J. D. (2006b). Test-enhanced learning taking memory tests

improves long-term retention. Psychological science, 17 (3), 249–255.

Rohrer, D. (2012, July). Interleaving Helps Students Distinguish among Similar Con-

cepts. Educational Psychology Review, 24 (3), 355–367.

Rohrer, D., Dedrick, R. F., & Burgess, K. (2014, October). The beneﬁt of interleaved

mathematics practice is not limited to superﬁcially similar kinds of problems. Psy-

chonomic Bulletin & Review, 21 (5), 1323–1330.

Rohrer, D., Dedrick, R. F., & Stershic, S. (2015). Interleaved Practice Improves Mathe-

matics Learning. Journal of Educational Psychology, 107 (3), 900–908.

Rohrer, D., & Taylor, K. (2007). The shuﬄing of mathematics problems improves

learning. Instructional Science, 35 (6), 481–498.

Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2011, April). Improving

students’ help-seeking skills using metacognitive feedback in an intelligent tutoring

system. Learning & Instruction, 21 (2), 267–280.

Rozenshtein, A., Pearson, G. D. N., Yan, S. X., Liu, A. Z., & Toy, D. (2016). Ef-

fect of Massed Versus Interleaved Teaching Method on Performance of Students in

Radiology. Journal of the American College of Radiology, 13 (8), 979–984.

Ruthsatz, J., Ruthsatz, K., & Stephens, K. R. (2014). Putting practice into perspective:

Child prodigies as evidence of innate talent. Intelligence, 45 , 60–65.

Schneider, V. I., Healy, A. F., & Bourne Jr, L. E. (2002). What Is Learned under Diﬃcult 75

Conditions Is Hard to Forget: Contextual Interference Eﬀects in Foreign Vocab-

ulary Acquisition, Retention, and Transfer. Journal of Memory and Language,

46 (2), 419–440.

Shaughnessy, J. J. (1977, January). Long-Term Retention and the Spacing Eﬀect in

Free-Recall and Frequency Judgments. American Journal of Psychology, 90 (4),

587–598.

Shea, J. B., & Morgan, R. L. (1979). Contextual interference eﬀects on the acquisi-

tion, retention, and transfer of a motor skill. Journal of Experimental Psychology:

Human Learning & Memory, 5 (2), 179–187.

Shea, J. B., & Zimny, S. T. (1983). Context Eﬀects in Memory and Learning Move-

ment Information. In R. A. Magill (Ed.), Diﬀering perspectives in motor learning,

memory, and control (pp. 345–366). North-Holland.

Shewokis, P. A. (2003). Memory Consolidation and Contextual Interference Eﬀects With

Computer Games. Perceptual & Motor Skills, 97 (2), 581–589.

Spruit, E. N., Band, G. P. H., & Hamming, J. F. (2015). Increasing eﬃciency of surgical

training: eﬀects of spacing practice on skill acquisition and retention in laparoscopy

training. Surgical Endoscopy, 29 (8), 2235–2243.

Stefanidis, D., Sierra, R., Korndorﬀer, J. R., Dunne, J. B., Markley, S., Touchard, C. L.,

& Scott, D. J. (2006). Intensive continuing medical education course training on

simulators results in proﬁciency for laparoscopic suturing. The American Journal 76

of Surgery, 191 (1), 23–27.

Sugiyama, M., Araki, M., & Choshi, K. (2006). Order of a ‘Uniform Random’ Presen-

tation on Contextual Interference in a Serial Tracking Task. Perceptual & Motor

Skills, 102 (3), 839–854.

Sweller, J., & Chandler, P. (1994). Why some material is diﬃcult to learn. Cognition &

Instruction, 12 (13), 185–233.

Taylor, K., & Rohrer, D. (2010). The eﬀects of interleaved practice. Applied Cognitive

Psychology, 24 (6), 837–848.

Tennyson, R. D., Steve, M. W., & Boutwell, R. C. (1975). Instance sequence and analysis

of instance attribute representation in concept acquisition. Journal of Educational

Psychology, 67 (6), 821–827.

Timby, B. K. (2016). Fundamental nursing skills and concepts (11th ed.). Lippincott

Williams and Wilkins. van den Boom, G., Paas, F., & van Merrienboer, J. J. G. (2007). Eﬀects of elicited

reﬂections combined with tutor or peer feedback on self-regulated learning and

learning outcomes. Learning & Instruction, 17 (5), 532–548.

Van Merrienboer, J. J. G., De Croock, M. B. M., & Jelsma, O. (1997). The Transfer

Paradox: Eﬀects of Contextual Interference on Retention and Transfer Performance

of a Complex Cognitive Skill. Perceptual & Motor Skills, 84 (3), 784–786. van Merrienboer, J. J. G., Kester, L., & Paas, F. (2006). Teaching complex rather than 77

simple tasks: balancing intrinsic and germane load to enhance transfer of learning.

Applied Cognitive Psychology, 20 (3), 343–352.

Van Patten, J., Chao, C.-I., & Reigeluth, C. M. (1986). A Review of Strategies for

Sequencing and Synthesizing Instruction. Review of Educational Research, 56 (4),

437–471.

Van Sickle, K. R., Ritter, E. M., Baghai, M., Goldenberg, A. E., Huang, I.-P., Gallagher,

A. G., & Smith, C. D. (2008). Prospective, Randomized, Double-Blind Trial of

Curriculum-Based Training for Intracorporeal Suturing and Knot Tying. Journal

of the American College of Surgeons, 207 (4), 560–568.

Wai, J. (2014). Experts are born, then made: Combining prospective and retrospective

longitudinal data shows that cognitive ability matters. Intelligence, 45 , 74–80.

Ward, P., Hodges, N. J., Starkes, J. L., & Williams, M. A. (2007, December). The road

to excellence: deliberate practice and the development of expertise. High Ability

Studies, 18 (2), 119–153.

Wayne, D. B., Barsuk, J. H., O’Leary, K. J., Fudala, M. J., & McGaghie, W. C. (2008).

Mastery learning of thoracentesis skills by internal medicine residents using sim-

ulation technology and deliberate practice. Journal of Hospital Medicine, 3 (1),

48–54.

Wayne, D. B., Butter, J., Siddall, V. J., Fudala, M. J., Linquist, L. A., Feinglass,

J., . . . McGaghie, W. C. (2005, July). Simulation-Based Training of Internal 78

Medicine Residents in Advanced Cardiac Life Support Protocols: A Randomized

Trial. Teaching and Learning in Medicine, 17 (3), 202–208.

Wayne, D. B., Butter, J., Siddall, V. J., Fudala, M. J., Wade, L. D., Feinglass, J., &

McGaghie, W. C. (2006, March). Mastery learning of advanced cardiac life support

skills by internal medicine residents using simulation technology and deliberate

practice. Journal of General Internal Medicine, 21 (3), 251–256.

Wayne, D. B., Didwania, A., Feinglass, J., Fudala, M. J., Barsuk, J. H., & McGaghie,

W. C. (2008). Simulation-Based Education Improves Quality of Care During

Cardiac Arrest Team Responses at an Academic Teaching Hospital: A Case-Control

Study. Chest, 133 (1), 56–61.

Welsher, A., & Grierson, L. E. M. (2017). Enhancing technical skill learning through

interleaved mixed-model observational practice. Advances in Health Sciences Edu-

cation, 22 (5), 1201–1211.

Whyte, J., & Cormier, E. (2014). A Deliberate Practice-Based Training Protocol for

Student Nurses: Care of the Critically Ill Patient: A Randomized Controlled Trial

of a Deliberate Practice-Based Training Protocol. Clinical Simulation in Nursing,

10 (12), 617–625.

Wilford, A. (2006, September). Integrating simulation training into the nursing curricu-

lum. British journal of nursing (Mark Allen Publishing), 15 (17), 926–930.

Wilson, B., & Cole, P. (1992). A critical review of elaboration theory. Educational 79

Technology Research & Development, 40 (3), 63–79.

Young, D. E., Cohen, M. J., & Husak, W. S. (1993, September). Contextual interference

and motor skill acquisition: On the processes that inﬂuence retention. Human

Movement Science, 12 (5), 577–600.

Zulkiply, N., & Burt, J. S. (2013). The exemplar interleaving eﬀect in inductive learning:

Moderation by the diﬃculty of category discriminations. Memory & Cognition,

41 (1), 16–27.