Systematic Review of Studies of Nursing Education Outcomes: an Evolving Review Nancy Spector, Phd, RN, Director of Education

National Council of State Boards of Nursing

Systematic Review of Studies of Nursing Education Outcomes: An Evolving Review Nancy Spector, PhD, RN, Director of Education

April 2006

The systematic review is an integral part of evidence-based health care. One of the best definitions of evidence-based medicine (which can be applied to health care in general) is “…the integration of best research evidence with clinical expertise and patient values” (Sackett, Straus, Richardson, Rosenberg, & Haynes, 2000, p. 1). This is a comprehensive definition that doesn’t just include the results of the best studies, but it also considers clinical expertise and the patients’ needs. When applying evidence- based health care to nursing education, we should employ the best studies available, integrated with the expertise of qualified and experienced nursing faculty and the values and needs of our students.

A systematic review is the overview of several randomized trials of the same intervention or treatment for the same situation or condition; this overview systematically and critically reviews and combines all the studies, providing a better answer than the results from just one study (Sackett et al., 2000). Since there are not a lot of randomized trials available on nursing education, this systematic review is intended to be a critical analysis of evidence supporting, or not supporting, nursing education strategies and learning environments. It is important to note that this is an evolving review that will continually change as more research becomes available.

Systematic reviews consider the strength of the evidence for a particular strategy. Therefore, in this review the levels of evidence, or hierarchies of the studies, are identified. There are several ways that researchers classify research studies. One system is to grade the studies on a rating of I to V. Level I studies are large randomized control trials (RCTs); level II studies are RCTs with 50 subjects or fewer; level III are smaller cohort or case-control and cohort studies; level IV evidence come from case reports and low-level case-control and cohort studies; finally, level V is expert or consensus based on experience, physiology or biological principles. Another system uses the levels A through D to designate the strength of the evidence. Grade A is the strongest evidence, while grade D is the weakest (Mayer, 2004).

Many systematic reviews only use randomized controlled trials; however, that would limit the results in this review. Therefore, in this review, the level of evidence will be rated as adapted from Gallagher (2003) and Polit & Beck (2004). Gallagher (2003), while writing a clinical article, used a meaningful, easily understood method of rating studies. To avoid confusion, Polit and Beck’s description of Level II nonexperimental studies was used to be more in line with nursing studies.

It is important for nurses to strongly consider the level of evidence when making decisions to use research in their practice. Level II or III evidence should not be discounted. If those studies are done well, they can begin to identify relationships, obtain information about populations and help us to understand the viewpoints and realities of those under study (Polit & Beck, 2004). Further studies can corroborate these findings or study the variables in a more controlled design. For the purposes of this review, the levels of the studies will be identified as: Level I. A properly conducted randomized controlled trial, systematic review or meta- analysis.

1 Level II. Other studies, such as quasi-experimental, correlational, descriptive, survey, evaluation, and qualitative. Level III. Expert opinions or consensus statements The databases used to retrieve these studies were CINAHL, Medline and ERIC. Keywords used were: education, nursing, teaching, education research, learning methods, learning strategies, research- based education, and outcomes of education. The Reference Librarian at Rush Medical Center School of Nursing assisted in identifying appropriate articles. All issues not available at the Rush University School of Nursing were ordered.

The following criteria were used to select the studies: Study of educational outcomes. Identification of a design. Sample description. Comparison being studied or objective of the study (for noncomparison studies). Reporting of results. English-only studies (including countries outside the United States).

References Gallagher, R. (2003). An approach to dsypnea in advanced disease. Canadian Family Physician, 49, 1611-1616. Mayer, D. (2004). Essential evidence-based medicine. Cambridge: University Press. Polit, D.F. & Beck, C.T. (2004). Nursing research: Principles and methods. Philadelphia: Lippincott Williams & Wilkins. Sackett, D. L., Strauss, S.E., Richardson, W.S., Rosenberg, W., & Haynes, R.B. (2000). Evidence- based medicine. London: Churchill Livingstone.

Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards

Angel, B.F., Duffy, N = 142 Structured format for Outcomes were Learning characteristics One of very few studies that Evidence supporting M. & Belyea, M. undergraduate Health Pattern acquisition of (e.g., age or previous measured critical thinking that supervised (2000). An junior nursing Assessment versus knowledge and degree) affected which during clinical experiences. clinical experiences evidence-based students, during unstructured format. development of critical teaching strategy was Psychometrics of the with qualified faculty project for the Fall of 1996. thinking skills. effective. measurement tools needed can improve the evaluating strategies critical thinking of 93% female Longitudinal, quasi- Older students and those to be cited. to improve students. experimental study, without a previous degree Students assigned to knowledge 86% white utilizing a pre-/post-test tended to benefit more from groups by a stratified Teaching strategies acquisition and Average age 24 design. the unstructured approach; random procedure. may be affected by critical-thinking ±5.5 younger students tended to student performance in A Case Study benefit more from a characteristics. nursing students. Questionnaire was structured approach. Journal of Nursing developed to measure Knowledge and critical Education, 39, 219- knowledge and to elicit thinking improved after a 228. characteristics of semester of faculty critical thinking. Level II supervised clinical experiences.

Armstrong, S., & Stratified They developed a Data collection Three themes were identified: Methodology to ensure This study was able to Muller, M. (2002). A sampling was quality audit system included interviews, Structure – human reliability of the data was validate that structure, value clarification on used to obtain and conducted a naïve sketches and resources, technology, strong. process, and quality within student and study to describe a document analysis. theoretical learning Data were collected from results/outcomes are nursing colleges in employer value clarification on important areas to Data analysis was facilities, practical learning multiple sources. Gauteng. Curationis, samples. quality within the evaluate for board done using a facilities and strategy. Data collection might have February, 52-68. Nursing Colleges in approval surveys. Lecturers were modification of Tesch’s Process – leadership, been more consistent Gauteng. Level II selected by content analysis educational program, across all subjects. purposive procedure. relationship and research. Tape recordings might have sampling. Guba’s model of Results/Outcomes – been used on all subjects. Funders were trustworthiness was community outreach, selected by used to ensure truth- products of the college, purposive value, applicability, organizational development sampling. consistency and and recognition. neutrality.

Babenko-Mould, Y., Convenience Examination of the Design — Pre-/post- Self-efficacy for students in Content validity only was This supports the Andrusyszyn, M., & sample = 42 differences in fourth- test, quasi- the intervention group was established. idea that clinical Goldenberg, D. fourth-year year baccalaureate experimental, not found to be significantly Cronbach’s alpha reliability practicum (2004). Effects of nursing students nursing students’ nonequivalent group. different from that of coefficients were calculated experiences, with computer-based at an Ontario self-efficacy (or students in the control qualified faculty, Theoretical framework for both instruments and clinical conferencing University. confidence) in group. increases nursing — Bandura’s theory of were acceptable. on nursing students’ carrying out nursing students’ level of Control group = self-efficacy. In both groups, students’ While online evaluations self-efficacy, Journal competencies, when confidence in all 27 Self-Efficacy for were rated positively by the of Nursing using the addition of Descriptive analysis nursing competence Professional Nursing students, there were no Education, 43(4), Online computer was also used to domains. Competences Instrument differences between the 149-155. intervention conference explore themes (SEPNCI) scores increased groups (traditional and Online learning need group = 15. discussions, versus regarding strengths Level II from pretest to posttest. traditional with CML) not be geographically using only traditional and challenges of 95% female; All students agreed (some regarding self efficacy for oriented. conference online learning. average age of strongly) that computer nursing competencies; the Online learning can discussions. 24 ± 5. conferencing enhanced study, therefore, really be a positive Examination of learning. cannot conclude that CML experience for strengths and Four strengths associated can contribute to increased nursing students, challenges of with computer conferencing confidence levels, though though further testing computer-mediated were connection, support, authors conclude this. must be done to learning (CML). learning and sharing. determine if it is as effective as traditional Two challenges of CML methods. were time and Internet access.

Benner, P. (2004). Three studies: 1978–1981 — Qualitative design Demonstrated that the This study is a 21-year review Students learn best Using the Dreyfus 1978–1981 — Delineate and (narratives, interviews Dreyfus Model is useful for of Dr. Benner’s studies on the when qualified faculty Model of skill 21 paired new describe & observations) understanding learning Dreyfus Model. Because of provides coaching, acquisition to characteristics of needs of students and that, the methodologies feedback and

3 Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards describe and graduates & nurse performance 1978–1981 — nurses. weren’t described in much reflection throughout interpret skill their at different levels interviews with Novice or first year of detail, and establishing nursing education. acquisition and preceptors; 51 of education and paired samples; education — operates from reliability and validity in this During the Novice clinical judgment in experienced experience. interviews and/or the perspective of qualitative study weren’t stage, learning is best nursing practice and nurse 1988–1994 — (1) participant inflexible, rule-governed addressed. These may have fostered by providing education. Bulletin clinicians; 11 describe nature of observations with the behavior. been addressed in the prior safe, clear directions of Science, new graduates; skill acquisition in rest of the sample. Advanced Beginner or new published studies. first with simulations, Technology & 5 senior critical-care 1988–1994, 1996– graduate — heightened followed by situated Society, 24(3), 188- nursing nursing practice 1997 — Small group awareness of feedback and learning experiences. 199. students (2) delineate the narrative interviews, frequently experience Level II 1988–1994 — practical individual interviews, anxiety and fatigue. 130 practicing knowledge and participant Competent or one to two ICU and embedded in observation. years in practice — decides general unit expert practice. what is important based on nurses 1996–1997 — past experiences. 1996–1997 — Extension of Proficiency or transitional 75 critical care above study, with stage to Expert — develop nurses inclusion of other the ability to let the critical-care areas. situation guide the nurses’ responses. Expert or phronesis (practical wisdom) — the integrated rapid response is the hallmark of expertise.

Bjørk, I.T. & Four nurses Development of Longitudinal Many omissions and faults Patient conditions could The general Kirkevold, M. employed in practical skills of qualitative study. with their performances vary from one videotape to assumption that (1999). Issues in different surgical postoperative Videotaped the were seen even on the the next; thus, affecting the experience leads to nurses’ practical skill units of two ambulation and nurses during the third videotape, after eight complexity of the skill mastery was development in the Norwegian dressing wounds. skill performance to 14 months experience performance. challenged by this clinical setting. Hospitals. and interviewed and 25 experiences with The authors didn’t address study. Journal of Nursing nurses and patients skills. inter-rater reliability or their Active reflection of Care Quality, 14(1), afterwards; The nurses associated coding system. one’s own experience 72-84. observations done learning with efficiency and The selection of the nurses is a premise for Level II three times with motor aspects of wasn’t discussed. experiential learning. three to five month performance. If reflection is not intervals. The nurses did not have fostered, Videotaped actions much guided experience, improvement will not were described and the units did not occur. impressionistically encourage collaboration. Similarly, guided and coded. experience or feedback from qualified practitioners is essential for improvement.

Buckley, K. M. Convenience Investigated A descriptive Students’ qualitative While instructors weren’t Students’ needs for (2003). Evaluation sample of 58 differences comparative study comments revealed both blinded to the format, all structure, instructor of classroom-based, students between the same method was used. positive and negative exams were multiple choice interaction and a Web-enhanced, and enrolled in course content Outcomes measured aspects of online which limited the bias. feeling of belonging Web-based distance three being delivered by were exam scores, instruction. Groups were not equal in must be addressed in learning nutrition consecutive traditional, Web overall course No differences were found size. the development of courses for nutrition and enhanced, and grades and standard in student learning Students had no choice in distance learning undergraduate health courses. Web-based course evaluations. outcomes. the instructional format, courses. nursing. Journal of formats. N=24 in Web-enhanced courses though assignment to Information Nursing Education, traditional Investigated were most popular. groups was not specified. concerning students’ 42(8), 367-370. lecture format; differences preferred learning Comments showed that Because of small number of N=23 in Web- regarding student styles and Level II possible sources of student cases in each group, it enhanced perceptions of motivations for satisfaction are student would take a large effect format; and experiencing learning should be profiles, learner size to find significant N=11 in Web- content via these solicited before characteristics, student differences. based format. three different selecting the form motivation and the Administration of the formats. and extent of Traditional four- communication process. computer exams and the year paper and pencil exams technology used in a baccalaureate was starkly different. course.

4 Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards

degree Authors attested to Online nursing program, adequate reliability and courses can be just second-degree validity of the course as effective as students, and evaluation tool, though no traditional lecture RN BSN statistics were provided; courses. students. exam psychometrics weren’t cited.

Epstein, R. M. & 195 relevant Propose a Used the MEDLINE Definition of “professional Data retrieval was done by Might provide good Hundert, E. M. citations. definition of database from 1966 competence:” the habitual one researcher. grounding for (2002). Defining and professional to 2001 and and judicious use of Use of MEDLINE only continued assessing competence. referenced lists of communication, negates use of unpublished competence study. professional Review current relevant articles for knowledge, technical studies and dissertations Provides the boards competence. JAMA, means for English-language skills, clinical reasoning, and studies in other with information about 287(2), 226-235. assessing it and to studies. emotions, values and professional disciplines. comprehensively reflection in daily Level I suggest new Excluded articles Criteria for selecting the assessing practice for the benefit of approaches of that are purely studies allowed for a variety professional the individual and assessment. descriptive, duplicate of types of studies to be competence in health community being served. reports, reviews, and included. care workers. opinions and position Common methods: Can it be generalized to statements. subjective assessment by nursing? supervisors, multiple- choice exams evaluating Future directions in factual knowledge and comprehensive abstract problem solving, assessments were and standardized patient provided. assessments of physical exams and technical communication skills. Few assessments use participatory decisionmaking measures; few reliably assess clinical reasoning, systems-based care, technology and the patient-doctor relationship; and few incorporate the perspectives of peers and patients.

Girot, E.A. (1995). Convenience Compared the Comparative Nonuniversity students Psychometrics on Beginning evidence Preparing the sample. students’ quantitative study defined “critical thinking” as questionnaires not supplied. that developing practitioner for Setting: perceptions of their with a qualitative the ability to analyze the Essence of questionnaires critical thinking skills advanced academic England. critical thinking component. written text, whereas the needed to be shared. requires time and study: The development in a university students viewed exposure to others 25 Semistructured Self-reports can be biased, development of traditional university it as having a direct who are seeking undergraduate questionnaires. and need to have critical thinking. setting and in a short relationship with their own similar goals. nursing corroborating evidence. Journal of Advanced course setting. practice. students in the Critical thinking Nursing, 21,387- control group; The university-educated should be integrated 394. 15 in students reported that they throughout the Level II intervention were more confident in educational process, group. their critical thinking skills, rather than to be questioned their practice taught in one short more; receptive to new course, and should ideas, express themselves be taught better, and were more collaboratively with flexible and less ritualistic education and in practice. service.

5 Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards

Greenhalgh, T. 12 prospective Outcomes of Systematic review of A failure of student Many of the studies had When evaluating (2001). Computer randomized learning with published studies, engagement can occur methodological problems, computer-assisted assisted learning in studies of computer with 200 potentially because of online glitches lacked statistical power, learning, the schools undergraduate medical students assistance. relevant studies. and “dead” hypertext links. had possible contamination of nursing should medical education. with objective, All 12 had Evaluation of unsupervised between the intervention observe unsupervised [Electronic Version]. predefined comparison groups. students attempting to gain and control groups, and had students from remote British Medical outcome criteria. access from remote sites sample attrition sites. Journal, 322(7277), should include observation. Systematic review Aim to use a variety 40-44. Adequately train these rigorously reviewed 12 of teaching strategies, Level I teachers, and they should randomized controlled including traditional be the senior teachers. trials, using a standard methods, along with retrieval method and computer assisted Barrier to computer- objective selective criteria. learning products. assisted learning is poor integration with other forms Can it be generalized to of learning. nursing students? Sharing of templates within, and even outside of universities, should be considered.

Ironside, P. M. 18 students Explored how Audio-taped, Themes Vague, unclear or conflict- In outcomes (2003). New and 15 teachers and unstructured Thinking as Questioning: ing interpretations arising education students pedagogies for teachers were students experience interviews face-to- Preserving Perspectival during analysis were are taught that they teaching thinking: interviewed, to enacting a new face or by telephone. Openness. clarified by referring back to are safe if they know The lived date. pedagogy, Narrative the interview texts. what the teacher told Participants were Practicing Thinking: experiences of Pedagogy, and this them to know; with Includes asked to “tell about a Preserving Fallibility and The research team students and article explains how this pedagogy teachers and time that stands out Uncertainty. analyzed the texts and the teachers enacting students from enacting this for you because it interpretations for thinking is necessary These themes describe narrative pedagogy. all levels and pedagogy offers new shows what it meant coherence, comprehen- for knowledge and how the teachers and Journal of Nursing types of nursing possibilities for to you to teach a siveness and thoroughness. theory application. students experienced Education, 42(11), schools. teaching and class using Narrative In the future being thinking in the context of 509-516. learning thinking. Pedagogy.” safe in practice might Narrative Pedagogy and require nurses to Level II Further probing: how Narrative Pedagogy think in ways that “What did that mean influenced their thinking. to you?” or “Can you persistently question Thinking as questioning give an example?” practice. involved persistently With this pedagogy to Questions were questioning the meanings keep students and intended to keep and significance of learning patients safe in participants engaged experiences and making nursing practice, in their stories visible that which had “not teachers ask the without directing been thought of before.” necessary questions, them to particular The shift is to bring and content and aspects or events. complexity and uncertainty knowledge is Verbatim into the classrooms and extended and transcriptions. clinical situations, inviting enhanced. Data analyzed using students to think about Research shows that Heideggerian nursing practice. nursing faculty often hermeneutical The emphasis shifts from tell students there is phenomenology. the student acquiring the “no one right answer” teacher’s perspective to the and that it “all student exploring multiple depends,” although perspectives. their pedagogical practices often reflect and reinforce the opposite. In this pedagogy clear and concise test items are constructed that focus on analytical thinking.

Issenberg, S.B., Initial pool of 670 Reviewed and Databases included Medical simulations facilitate Rigorous systematic review Simulations are McGaghie, W. C., studies, reduced synthesized existing ERIC, MEDLINE, learning under the right with an eight-step pilot valuable learning Petrusa, E. R., to 109. evidence in PsychINFO, Web of conditions by: project; methodological experiences when

6 Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards

Gordon, D. L., & educational science Science and Timelit Providing feedback issues were attended to; carried out under the Scalese, R. J. to answer what 91 search terms and Repetitive practice then the six-step study right conditions. (2005). Features features and uses of phase was undertaken. concepts in their Curriculum integration and uses of high- high-fidelity medical Boolean All coding decisions were Range of difficulty fidelity medical simulations lead to combinations. unanimous, and each rater simulations that lead the most effective Multiple learning strategies Hand searching was blind to the coding to effective learning: learning. Capture clinical variation decisions of his/her partner. A BEME systematic Internet searching Controlled environment Much variation in the review. Medical Attention to “gray” Individualized learning strength of the findings in Teacher, 27, 10-28. literature Defined outcomes the peer-reviewed Level I Use of stringent publications. criteria for inclusion Simulator validity and Limits on the published of studies. effective learning correlate body of evidence ruled out Qualitative data formal meta-analysis. synthesis and tabular presentation of methods and outcomes.

Joubert, A., Viljoen, Convenience Evaluated the effect A quasi-experimental Computer teaching made Report was disorganized When used M. J., Venter, J. A., sample of 120 of the computer- design, using a no difference in and hard to read. appropriately, & Bester, C. J. generic nursing based teaching pretest/post-test respondents’ knowledge of Reliability and validity done computer-based (2002). Evaluation students in two program on control group problem on the instruments was teaching can increase of the effect of a educational knowledge, problem- Research was solving. relatively weak. knowledge. computer-based institutions in solving skills and conducted in a Computer teaching had a Some results weren’t Computer-based teaching programme South Africa. learning approach in nursing practice better effect in controlled included because partner teaching should not (CBTP) on relation to oncology setting (control) and circumstances. data was incomplete. be used alone, but it knowledge, content. under strictly should be used with The students were weak at There were basic problem-solving and controlled actual clinical identifying potential differences in the time learning approach. circumstances in a experiences. problems, and the spent using computer- Health Sa multimedia computer computer-based teaching based teaching between Gesondheid, 7(4), center and in a did not promote this ability. the intervention groups and 80-97. practice and Recommended that the control group. multimedia center Level II computer-based teaching (experimental Unclear how assignment to not be used in isolation. groups). groups was made. They used six Sample size was calculated instruments; the using multiple factors. reliability and validity Assessments were of each were comprehensive. addressed.

Kyrkjebø, J. M., & 44 first-year Objective was to Students participated Personal improvement Self-reports can be biased. Possible method of Hanestad, B.R. nursing students, describe the use of a in a two-hour session project seems to be an Important to develop teaching continuous (2003). Personal sample selection personal introducing them to the effective way of introducing instruments that can quality improvement. improvement project methodology was improvement project personal improvement continuous quality measure change in in nursing not documented. for teaching nursing project. They then improvement knowledge to knowledge. education: learning students about participated in nursing students. Excellent review and methods and tools continuous quality counseling sessions in Even those who did not integration of the literature. for continuous improvement. week three and seven, succeed in achieving a No reports of psychometrics quality improvement which lasted for one personal improvement felt on their questionnaire. in nursing practice. hour. At eight weeks they had a positive learning [Electronic Version]. the students gave a outcome. Journal of Advanced 10-minute presentation Teachers’ involvement in Nursing, 41(1), 88- of their projects. Data the program is important. 98. were collected using a questionnaire. Level II

Maag, M. (2004). A convenience Nursing students Described as Interactive media Further research is needed No significant The effectiveness of sample of ninety- were randomly experimental presentation of remedial to determine if increased differences were an interactive six assigned to one of multifactorial study, math and calculation learning can be achieved by shown, either with multimedia learning undergraduate four treatment though the sample concepts did not render providing multimodal online math scores, math tool on nursing nursing students, groups, all with wasn’t randomly statistically significant learning modules that efficacy, or students’ math attending two computer-based selected. Instruments: increases in mean math- nursing students can use at satisfaction with the knowledge and self- west coast learning methods: three investigator test scores or math efficacy their convenience for longer teaching strategy,

7 Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards efficacy. CIN: universities text only, text and designed criterion- at the post treatment and periods of time. with four different Computers, participated in the image, multimedia based tests involving follow-up treatment The investigators computer-based Informatics, Nursing, study. The and interactive basic math problems periods. acknowledge that their methods of 22(1), 26-33. students were multimedia. and drug calculations; Results indicated that a results are limited because instruction. mixed gender, the Mathematical Self Level II one-hour intervention is not of the short treatment time, While there was no ethnically diverse Efficacy Scale (MSES); sufficient to correct the a lack of strong student comparison with face- and their ages Student Satisfaction deep-seated math problem motivation, and the use of a to-face learning, ranged from 19 to Survey, which was that has been documented small convenience sample. these methods should 42 years. investigator designed. by educators for many Reliability and validity was continue to be The scales were given years. established on all the explored as useful at intervals, as The study showed that the instruments they used. adjunctive teaching described. The results computer-based learning methodologies. were analyzed with It would have been modules did not impede the descriptive statistics, instructive were they to students’ learning. one-way analysis of have had a fifth comparison covariance and one- Interactive multimedia group with face-face way analysis of group students were more learning. variance. satisfied with this method of Extraneous variables, such learning, though this as motivation, test anxiety, difference was not and computer attitudes significant. might be the focus in future studies.

MacIntosh, J., Four faculty They studied the first Longitudinal, Overall theme of Some distant professors There may be MacKay, E., Mallet- members, one at class of students phenomenological colearning: Main strength teaching by teleconference benefits of dividing Boucher, M., & each distance that entered an study. Data collection of colearning was “having a had less effective contact students in larger Wiggins, N. (2002). site and two at expanded program involved open-ended small class where you with learners; participants sites into smaller Discovering the main site, in in 1995 and 1999 questionnaires, really get to know and work indicated that this did groups to foster colearning with Canada. when they interviews, and focus with everyone. It’s really influence development of supportive students in distance Approximately 90 graduated. They groups. Common close-knit and everyone colearning relationships. interactions present in education sites. learners focused on themes were encourages you to do your Learners came to like smaller groups. Nurse Educator, participated understanding the generated by best”. computers because e-mail Interactions with 27(4), 182-186. yearly from three phenomenon of clustering similar Being able to study in provided a link to, and professors remain an sites. becoming nurses codes. There were Level II relatively small centers, interaction with, others, important part of within the context of approximately 12 to 18 with small numbers of including professors. learning. a curriculum that is audiotaped interviews learners and faculty, Some nonnursing courses oriented toward across the sites. created a family-like are more difficult to tele- caring and co- atmosphere that tended to conference and teachers learning and that is support learning. accustomed to lecturing delivered across Findings indicate that content on campus made three geographically orientation to a multisite learners question the distant sites. program must include effectiveness of this familiarization with distance teaching strategy. technologies for both The required group projects students and teachers. to build colearning Interaction with profes-sors contributed to a sense of was important for work overload. colearning. Reliability of transcriptions wasn’t addressed.

McDonald, D. D., The sample size They tested how A pre-/post-test, The results suggest that The study took place in a Greater effort should Wiczorek, M., & started with 78 background noise double-blind, two-by- noise and interruption university rather than in a be made to create Walker, C. (2004). college students. and being two factorial during health teaching health care setting. Health environments with Factors affecting The final sample interrupted affect experiment adversely affects the ability care and university minimal distraction, learning during size was 48 (see learning health comparing to learn health information. environments might especially when health education strengths and information. interruption The difference in the mean introduce vastly different understanding the sessions. Clinical weaknesses; (interruption/no scores was small, but a intrapersonal factors that health information, is Nursing Research, third bullet). interruption) by noise lack of understanding in encourage or inhibit critical. 13(2), 156-167. (noise/no noise) was learning. Average age – any one of the areas could People teaching used. Level II 21.4 (± 6.21) place a person at risk. The study controlled for health information Instruments included confounding factors, such should assess the White – 79.5% a demographic data as different teachers, environmental Female – 83.3% record and the content, frequency, and distractions present, Antibiotic Resistance magnitude of disruption. develop plans to Nursing major – Test, where content The groups, though formed decrease the factors,

8 Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards

71.8% validity and reliability with random selection, were complete a cost- were established. uneven regarding having benefit analysis for The teaching taken a microbiology each option, intervention was a course; therefore, the implement changes, five-minute original findings showed no and evaluate the videotape on differences. They found effectiveness of the antibiotic-resistance differences when they changes for health- teaching, and omitted those students who learning outcomes. content validity was had taken a microbiology established. course. However, that decreased their sample size Participants were from 78 to 48. randomly assigned to one of the four The distractions and noise groups. were realistic. A data recall task The teaching was only done was presented by videotape, thereby immediately after the negating teacher/student videotape, which interaction, which could required the clarify misperceptions. information learned while watching the videotape to be transferred into long- term memory to be recalled. Analysis of covariance was used in analysis.

Miller, S. K. (2003). Convenience The medical The study design Student satisfaction scores Caution is advised using This pilot study A comparison of sample of 12 literature has studied was experimental, showed no significant such a small sample size. suggests that student outcomes APRN students in problem-based post-test only, differences between the The effect size would have problem-based following problem- the control group learning more though the sample groups. had to have been large to learning may be at based learning and 10 APRN comprehensively wasn’t randomly Midterm exams showed no have shown significance. least as effective as instruction versus students in the than nursing. selected. significant differences One intervening variable traditional lecture and traditional lecture intervention Therefore, this study They cite that they between the groups. that the researchers should be explored in learning in a group. compared student larger studies. didn’t need a pretest Final exam grades showed acknowledged was that the graduate performance and because it was a no significant differences intervention group were not The evidence did not pharmacology satisfaction in homogenous between the groups. only learning new material, support problem- course. Journal of problem-based sample. but a new learning method based teaching the American learning to a The same faculty at the same time. This methodologies over Academy of Nurse traditional lecture member taught each could have affected the traditional Practitioners, format in class. results. methodologies. 15(12), 550-556. pharmacology. The students were Since problem-based Level II blinded to the fact learning is thought to that another teaching improve critical thinking, the method was being difference might be seen in used, and the groups practice, rather than with were 50 miles apart the exams. from each other. The psychometrics of the The Student exams were not provided. Satisfaction with Can the results be Learning Tool had generalized to respectable content undergraduate nursing validity and test- students? retest reliability. No psychometrics were supplied for the midterm exam and final exams. The Students’ t tests for independent samples were used for analyzing

9 Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards

differences. The teacher did not know whether she was grading a control or experimental exam.

Murphy, M. (1995). Participants for Describe the Qualitative study Both practice and Lack of clarity of definition Educators and Open learning: the this study were feelings and using guided, educators valued open- within both groups as to practitioners saw managers’ and from a college of motivations of nurse standardized learning as a mode of what exactly open-learning clinical experiences educationalists’ nursing and its educators and interviews with an program delivery is. as vital in the perspective. clinical links. managers toward open-ended, in- appropriate for a practice Sample selection process education of nurses, [Electronic Version]. open-learning depth interview profession. and open-learning Setting was was not made clear. Journal of Advanced programs. technique. would only be a part England. Both groups confused the Line-by-line coding of Nursing, 21(5), of teaching nursing The definition they Tape-recorded concepts of open and interview transcripts 1016-1023. students. used was that open interviews were distance learning. allowed for comprehensive Level II learning relates to transcribed verbatim. All interviewees agreed that results. Open-learning is often confused by an educational Situation analysis open learning would help to Researcher acknowledged nurse educators and philosophy where was used, requiring close the theory-practice that some would use the managers as being the learners have detailed, searching gap. survey method, and yet she distance-learning. access not just to and concrete The interviewees saw open cogently argued that the educational analysis of the data learning as a way of survey method hands over products, but to the collected to ‘get empowering the learner. the data collection from the means of shaping inside the The findings showed that researcher to the informant. their own learning. information.” practice and education Researcher acknowledged A theoretical aren’t working the lack of rigor with open framework was collaboratively, but each interviews, and yet she devised that are functioning with their argued that a rigid interview combines the own competitive market in could be dominated by the philosophies of mind. researcher’s agenda. humanistic education and Knowles’ andragogical assumptions for learning with concepts of student empowerment and increasing clinical competence.

Platzer, H. Blake, 30 students were Develop a better Groups were The students reported Self reports can be biased. Excellent qualitative D., & Ashford, D. followed for two understanding of the qualitatively significant development in No measurements of critical evidence to support (2000). An years in England. use of groups and evaluated by the use their critical thinking thinking were made. the need for students evaluation of discussions to of in-depth, abilities. to reflect in groups The reliability of the coding process and facilitate reflective semistructured and discussions The reported greater and categorization was not outcomes from practice. interviews. about their practice. autonomy to question the discussed. learning through Interviews were status quo. reflective practice audio-recorded and The participants reported a groups on a post- transcribed. They less rule-bound approach registration nursing were analyzed using to their practice (relates to course. [Electronic a qualitative software Benner’s work). Version]. Journal of analysis package Their learning in the Advanced Nursing, (QST NUD-IST reflective practice groups 31(3), 689-695. version 3). can best be understood in Level II The data were coded terms of an increase in and categorized as professionalism. themes emerged.

Schaefer, K. M. & Sample consisted Describe the Descriptive Participants were more Nice literature review. Excellent suggestions Zygmont, D. (2003). of 178 females predominate correlation design teacher than student Investigators met to achieve were given for Analyzing the and nine males. teaching styles of with triangulation of centered; their written consensus about themes assisting faculty to teaching style of nursing faculty as methods was used. philosophies revealed both move to a more Mean age of 50. with the narrative data. nursing faculty: either teacher- teacher-centered and student-centered Principles of Adult Questionable Does it promote a centered or student-centered environment, and Learning scale generalizability beyond student-centered or student-centered. (PALS) was used to approaches. perhaps the boards of

10 Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards teacher-centered Compare faculty measure teacher or Faculty used student- baccalaureate nursing nursing could support learning teaching styles to student centered language in their programs accredited by the these. environment? their instructional centeredness; good philosophies, often in a NLNAC. It is helpful for faculty Nursing Education methods and to reliability and validity teacher-centered context; to regularly share Perspectives, 24(5), their stated have been reported therefore, they may have effective teaching 238-245 philosophies. for this tool. recognized the need for a methods with their student-centered Level II Questionnaires were peers in formal and sent to 100 randomly environment but had informal settings. selected difficulty with baccalaureate implementation. programs accredited A distinction was made by NLNAC to be between clinical and given by the dean to classroom teaching. The five faculty members. authors questioned whether the philosophy should change according to the teaching venue.

Simmons, B., 15 experienced Explored cognitive Qualitative, The most common thinking Three nurses spoke English While this sample Lanuza, D., nurses. strategies used by descriptive design. strategies used to reason as a second language, was of experienced Fonteyn, M., Hicks, experienced nurses about assessment findings which may have affected nurses, it provides Five adult med- The think-aloud F., & Holm, K. as they considered were recognizing a pattern, understanding and thinking insight for teaching surg units in a method was used to (2003). Clinical assessment findings judging the value, providing aloud. strategies with clinical teaching assess cognitive reasoning in of assigned patients. explanations, forming reasoning and critical community processes. Few studies of nurses’ experienced nurses. Experienced nurse relationships, and drawing thinking. hospital outside a Participants were clinical reasoning have Western Journal of was defined as conclusions. large Midwestern tape recorded by an been conducted in a The think-aloud Nursing Research, practicing from two city. investigator to gather Nurses made sense of practice setting during method is an effective 25(6), 701-719. to 10 years, full-time, information. assessment information by actual patient care. way to access the Convenience on a medical- linking concepts together to cognitive processes Level II Each audiotape was The number of years in sample. surgical unit. form relationships. used in clinical transcribed. The text practice may not have been reasoning and might was methodically These relationships an appropriate indicator of be used by faculty reviewed using the indicated the specific skill level, and the authors teaching nursing three steps of information nurses were acknowledge this. students. protocol analysis: concentrating on and referring phrase determined the direction Although previous analysis, assertional that their reasoning would research has shown analysis, and script take next. that expert nurses analysis. Years in practice is only chunk information one criterion to distinguish and employ thinking between nursing skill level. strategies to speed the reasoning process, this study indicated that experienced nurses (who were not experts) employed similar techniques

Thiele, J. E. (2003). The sample The objective was The study was These results indicate that Methodology poorly Additional research Learning patterns of consisted of 64 to learn how online conducted during the learned information described. with controlled online students. students in a courses affect two sequential component was higher for the The comparison of an RN- comparisons between Journal of Nursing baccalaureate learners. semesters. online students than for the BSN group to a generic traditional and online Education, 42(8), program for RN The researcher During each traditional students. group is inherently flawed. courses is needed to 364-366. students that expand the also compared semester, three face- The exam procedures were completed a knowledge of the Level II students’ learning to-face class starkly different: The RN- three-credit effects of Web-based in a traditional meetings were BSN students took the research and education on learners course to those in conducted. The exam online at home, with informatics and learning an online course. remaining 12 classes no time period; no controls course. outcomes. were conducted in were placed on the Learning an asynchronous students, except for Results of this study outcomes were format (via e-mail, personal integrity. should be used very compared to 42 group discussion Meanwhile the generic cautiously because of generic board or telephone). students took a 50-minute the methodological students. All assignments were proctored exam. concerns. posted online and K-R reliability for the exam

11 Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards

required use of Web was acceptable; no validity resources for data were provided. completion. Learning outcomes were measured with online exams. Generic students were taught the same content with a traditional methodology and took an “almost identical” exam.

Tiwari, A. & Tang, The sample The purpose was to The study involved Three themes emerged from Self report data can be Backwash was C. (2003). From consisted of 70 evaluate the the use of a the analysis of the interview biased. discussed, and it process to outcome: nursing students effectiveness of nonequivalent transcriptions: Intervention was only one implies that what the the effect of portfolio in the portfolio assessment control group design, students learn and The students favored the semester. assessment on Department of in enhancing student as well as a how they learn use of portfolio Other incidental variables student learning. Nursing Studies learning. qualitative depends very much assessment. could have been a factor, Nurse Education in The University component. on what they think This Paper mainly The process of preparing as the authors Today, 23, 269-277. of Hong Kong. Data collection they will be assessed addressed the portfolios yielded positive acknowledge. consisted of the on; this seems to be Level II 21 second year qualitative data. academic and affective This was only the report of Study Process very appropriate for students were outcomes. the qualitative results; the Questionnaire (SPQ) those nurse assigned to the quantitative results can be to measure students’ Unexpected findings in the educators who “teach treatment found: Tiwari, A. & Tang, C. approaches to form of spontaneous to” the NCLEX. group. (2001). The power of learning; the collaborative learning The positive 49 third year partnership: Enhancing Assessment during the process of academic and students were student learning through Preparation Strategy preparing portfolios for affective outcomes assigned to the assessment by portfolio. In: Questionnaire those students who lacked suggest that the use comparison Conway, J. (ed). Research (APSQ) to find out motivation. of portfolios can have group. & Development in Higher how students The positive academic a positive effect on Education: Vol. 24. prepare for their outcomes were (supported by learning. Learning Partnerships. The assessments; and qualitative comments): Higher Education Research semi-structured Gaining a much better and Development Society interviews with understanding. of Australasia, Inc., ACT, selected students Applying what they learn to pp. 188-194. (12) to explore their their professional practice. experience and perceptions of the Learning deeply and assessment process. meaningfully. The audio-recordings Conceptualizing at a high were transcribed cognitive level. verbatim to improve Gain in confidence. trustworthiness. Pleasure, appreciation and freedom to choose.

White, A. H. (2003). Seventeen senior They studied the A qualitative Five themes were identified Besides using the constant The importance of Clinical decision nursing students essential methodology was as components associated comparison method to students working with making among (16 women, one components of used, Heideggerian with nursing students’ clinical identify themes, the staff was paramount. fourth-year nursing man) participated learning clinical phenomenology, with decision making: investigators asked three When students students: an in the study. No decisionmaking hermeneutical Gaining confidence in their students to verify the gained confidence, interpretive study. students had any among nursing analysis. skills. themes. they were able to shift Journal of Nursing type of previous students. The identification of Building relationships with A model was presented to their focus from Education, 42(3), degree in themes was staff. encourage further dialogue. themselves to the 113-120. nursing. All accomplished A more diverse student clinical environment. students were Connecting with patients. Level II through the accepted sample would have made When students completing their Gaining comfort in self as a constant the study more worked in the clinical last semester of nurse. comparative method. generalizable. This should area with patients and coursework in a Understanding the clinical be considered for future mentors, they gained baccalaureate picture. research. in confidence. program. The results of the study When students indicate that until students became more are able to understand the comfortable in the

12 Article Sample Comparison Study Key Strengths & Implications Studied Procedures Results Weaknesses for Boards

clinical picture, their clinical clinical environment, decision making capabilities they began to are limited. assume the nursing role. The researcher questions whether traditional clinical Until students rotations are as effective as a understand the consistent clinical clinical picture, their experience. clinical decisionmaking capabilities are limited. Yet, they need knowledge, experience and self- confidence to understand the clinical picture.

Yates, P. Jackie, C. 55 of 323 first Examine the Five sessions of one The program was Evaluations were Students feel it is Moyle, W. & Wollin, year students potential of peer to two hours’ perceived to provide a comprehensive. important to integrate J. (1997). Peer enrolled in the mentorship to assist duration were held considerable amount of Because the protégés were both theory and mentorship in Bachelor of students to improve every two to three help to participating volunteers, there may have practice. clinical education: Nursing program their clinical learning weeks during the 14- students, particularly in been a systematic bias. Most clinical teachers outcomes of a pilot agreed to outcomes. week semester. reducing anxiety and agree that lack of programme for first participate. Eight Sessions focused increasing confidence. confidence and year students. peer mentors specifically on There were no differences anxiety can have Nurse Education were selected strategies for between the groups related detrimental effects on Today, 17, 508-514. from students in negotiating the to clinical instructor ratings. student learning, and the second year Level II clinical environment, Mentors felt the program the strategy of using of the program to promoting learning had assisted students with peer mentors may facilitate the from clinical increasing confidence and assist with this. sessions. The experience, and reduced anxiety. Support strategies, setting was debriefing of events which reduce stress Australia. 55 Students were less and experiences for beginning randomly satisfied with issues such during clinical students are selected non- as timing and organization practicums. important in nursing participants of the sessions. Measurement programs, since they served as the Students spoke of their included pre- and are likely to contribute controls. concerns about the need post-program to an improvement in for practice of clinical skills questionnaires, a student performance to improve their confidence focus group and a decrease in and reduce anxiety. interview, review of student attrition. mentor journals, and a statistical analysis of the differences in clinical ratings between the participants and non- participants.

13 Conclusion This is an ongoing project where we are continuing to search for studies that meet the specified criteria. A limitation of any systematic review is that it is only good as the quality of research that it covers. As discussed in strengths and weaknesses, oftentimes sample sizes were small and controls were lacking. The study criteria for this systematic review were not as stringent as some reviews so that the breadth of the literature could be reviewed. The review identifies strengths and weaknesses of the studies so that the reader can decide how to use these findings.

Three Level I systematic reviews were identified. Epstein & Hundert (2002) defined “professional competence” and provided some guidance for boards for assessing the competence of health care workers. Greenhalgh (2001) identified 12 prospective randomized studies of medical students for the purpose of evaluating computer-assisted learning. They suggest that computer-based learning can be effective, though the aim should be to use a variety of teaching strategies. Issenberg et al. (2005) conducted a systematic review of high-fidelity medical simulations for learning and found them to be valuable adjuncts to learning when carried out under the right conditions.

Five of the studies provided evidence that qualified faculty were important for teaching nursing students, though there was no literature about specific qualifications. Two studies specifically identified the need to improve students’ confidence levels before they can effectively think critically when caring for patients. Five studies provided evidence that clinical experiences improve students’ abilities to think critically when caring for patients, though there were no studies found that investigated specific numbers of clinical hours. Likewise, there were no studies that evaluated those programs that do not have, or have very limited, clinical experiences. Two studies found that reflective practice was a very important strategy for teaching nursing students to critically think. There were four studies that showed no differences in learning outcomes with online courses versus traditional courses, and one found online courses had significantly better student outcomes, though that particular study was not well controlled and should be replicated. Other research investigated some very specific issues, including: Validating the need to evaluate structure, process and results/outcomes when evaluating programs. Validating personal improvement courses for teaching continuous quality improvement. Decreasing environmental noises and distractions in order to enhance learning. Problem-based learning, compared to traditional learning, was investigated. Provided good guidelines for assisting the faculty members in moving towards a more student- centered way of teaching. Supported the IOM’s recommendation for the importance of interdisciplinary teams in health care. Supported the use of portfolios for student learning assessment. Recommended the “think aloud” strategy for clinical reasoning. Supported peer mentors as a way to increase student confidence levels in clinical situations.