THE RELATIONSHIP OF SELF-DIRECTED LEARNING READINESS TO
KNOWLEDGE-BASED AND PERFORMANCE-BASED MEASURES OF SUCCESS
IN THIRD-YEAR MEDICAL STUDENTS
by
Brian W. Findley
A Dissertation Submitted to the Faculty of
The College of Education
in Partial Fulfillment of the Requirements for the Degree of
Doctor of Philosophy
Florida Atlantic University
Boca Raton, FL
August 2009
Copyright by Brian W. Findley 2009
ii
THE RELATIONSHIP OF SELF-DIRECTED LEARNING READINESS TO
KNOWLEDGE-BASED AND PERFORMANCE-BASED MEASURES OF SUCCESS
IN THIRD-YEAR MEDICAL STUDENTS
by
Brian W. Findley
This dissertation was prepared under the direction of the candidate’s dissertation advisors, Dr. Lucy M. Guglielmino and Dr. John D. Morris, Department of Educational Leadership, and has been approved by the members of his supervisory committee. It was submitted to the faculty of the College of Education and was accepted in partial fulfillment of the requirements for the degree of Doctor of Philosophy.
SUPERVISORY COMMITTEE:
______Lucy M. Guglielmino, Ed.D. Dissertation Advisor
______John D. Morris, Ph.D. Dissertation Advisor
______Michele Acker-Hocevar, Ph.D
______Valerie C. Bryan, Ed.D. ______Robert E. Shockley, Ph.D. Chair, Department of Educational Leadership
______Valerie J. Bristor, Ph.D. Dean, the College of Education
______Barry T. Rosson, Ph.D. Date Dean, Graduate College
iii
VITA
Brian W. Findley graduated from Slippery Rock University of Pennsylvania in
1983 with a B.S. in Exercise Science. He moved to South Florida that same year, working as a clinical exercise physiologist and assuming leadership positions in hospitals, outpatient physical rehabilitation clinics, and corporations. After receiving his M.Ed. from Florida Atlantic University in Exercise Science and Wellness Education in 1993, he began teaching as an adjunct professor at the community college level. A full-time position as a professor at Palm Beach Community College allowed him to move into educational leadership positions; first as department chair and cluster chair, and then as an interim associate dean. In 2006, he was named PBCC Professor of the Year by his peers. Findley received his Ed. S. in 2007 and Ph.D. in 2009, both in Educational
Leadership with a specialization in Adult and Community Education, from Florida
Atlantic University. He is a charter member of the International Society for Self-Directed
Learning. Findley has authored 5 book chapters, 14 peer-reviewed research articles, and
68 abstracts. Throughout his career, he has made over 85 presentations of original research at regional, national, and international conferences. Findley currently resides in
Royal Palm Beach, Florida, with his son David.
iv
ACKNOWLEDGEMENTS
“I can do all things in Christ, who strengthens me” (Philippians 4:13).
I have been blessed to have a “community of scholars” that share in my academic
success at Florida Atlantic University. First and foremost, I would like to thank my
academic advisor and dissertation co-chair, Dr. Lucy M. Guglielmino, for her guidance,
support and dedication. Dr. Guglielmino’s tireless efforts and constant encouragement have made the greatest difference in my academic success. My sincere thanks go to co-
chair, Dr. J. Dan Morris, and committee members, Dr. Michelle Acker-Hocevar, and Dr.
Valerie Bryan, who all gave selflessly of their time and energy. From the challenging
coursework, to the comprehensive examination and throughout the dissertation process, this amazing group of faculty has inspired me and believed in me. My deep appreciation goes to Dr. Robert Bulik for allowing me access to UTMB and the collected data. His generosity will never be forgotten.
I owe so much of what I have learned about research and professionalism to my friend and colleague, Dr. Lee E. Brown. All of my doctoral student colleagues at FAU, particularly Dr. Darwin E. Asper, Gerri Penney, and Dr. Jo Ann Bamdas have been
instrumental to the completion of this project.
My church family at Christ Fellowship and my work family at Palm Beach
Community College have been incessantly supportive and understanding.
v Finally, my family has been an integral part of this success. My parents have supported me, inspired me and loved me each step of the way. My son, David, has been my greatest fan and I thank him for helping me overcome the stressful times by allowing me to see the world through his eyes.
vi
ABSTRACT
Author: Brian W. Findley
Title: The Relationship of Self-Directed Learning Readiness to Knowledge-Based and Performance-Based Measures of Success in Third-Year Medical Students
Institution: Florida Atlantic University
Dissertation Advisors: Dr. Lucy Madsen Guglielmino and Dr. John D. Morris
Degree: Doctor of Philosophy
Year: 2009
The purpose of this study was to investigate the self-directed learning (SDL)
readiness of third-year medical students in comparison to previously reported scores for
the general population; the relationship between SDL readiness and knowledge-based and
performance-based measures of success in a medical school using an integrated medical
curriculum; and to determine if knowledge-based and performance-based measures of
success are significant in predicting Self-Directed Learning Readiness Survey/Learner
Preference Assessment (SDLRS/LPA) and National Board of Medical Examiners Family
Medicine Shelf Examination (NBME-FM) scores. This study analyzed SDLRS/LPA scores, knowledge-based scores (NBME-FM), performance-based scores (Objective
Structured Clinical Examination [OSCE] and preceptor rating), and a combination of knowledge-based and performance-based scores (final grade).
vii Analyses of 873 students resulted in mean scores of 229.06 + 23.19 for the
SDLRS/LPA. Correlations were significant (p < .05) for SDLRS/LPA scores to NBME-FM
scores (r = .073, p < .05). OSCE scores (r = .133, p < .01), and final grade (r = .138, p <
.01). Regression analysis revealed that the total model of NBME-FM, OSC AVG, and
preceptor rating predicted 2.1% of the variation in SDLRS/LPA, which was significant (p
< .01). Regression analysis revealed that SDLRS/LPA, OSC AVG and preceptor ratings
predicted 9.7% of the variance in NBME-FM, which was significant (p < .001).
The results support previous findings that medical students’ levels of SDL
readiness are higher than the general population mean of 214.0 + 23.49. While the
SDLRS/LPA scores of medical students with knowledge-based and performance-based
examinations were modest, they mirror the relationships that have appeared consistently
across a number of studies and indicate a tendency for students with higher levels of SDL
to perform better in medical preparation programs. The SDLRS/LPA adds an important dimension to the holistic assessment of medical students, addressing the emphasis on ensuring that physician preparation programs produce practitioners who are likely to be continuing, lifelong learners. This investigation of SDL in medical education was unique in that it may be the first to look at the relationships of SDLRS/LPA scores with both knowledge-based and performance-based measures as well as with a combination of knowledge-based and performance-based measures.
viii
To David, Mom, and Dad
THE RELATIONSHIP OF SELF-DIRECTED LEARNING READINESS TO
KNOWLEDGE-BASED AND PERFORMANCE-BASED MEASURES OF SUCCESS
IN THIRD-YEAR MEDICAL STUDENTS
LIST OF TABLES...... xiii
LIST OF FIGURES ...... xiv
INTRODUCTION ...... 1
Rationale of the Study...... 4
Problem Statement...... 5
Purpose of the Study...... 6
Research Questions...... 7
Hypotheses...... 8
Significance of the Problem...... 9
Definition of Terms...... 10
Limitations and Delimitations...... 16
Limitations ...... 16
Delimitations...... 16
Organization of the Study ...... 17
LITERATURE REVIEW ...... 18
Overview: Physicians and Physician Preparation...... 18
Integrated Medical Curriculum...... 19
ix Problem-Based Learning/Evidence-Based Medicine (PBL/EBM) ...... 21
The National Board of Medical Examiners Assessments...... 26
History of The Objective Structured Clincial Examination (OSCE)...... 28
Self-Directed Learning (SDL) ...... 33
Self-Directed Learning in the Medical Field ...... 40
Chapter Summary ...... 43
METHODOLOGY ...... 44
Institutional Setting for the Study...... 45
Subjects ...... 49
Instrumentation ...... 50
Self Directed Learning Readiness Scale/Learning Preference Assessment (SDLRS/LPA) ...... 50
National Board of Medical Examiners Family Medicine Shelf Exam (NBME-FM)...... 52
Objective Structured Clinical Examination (OSCE) ...... 53
Preceptor Rating...... 54
Final Grade...... 54
Procedures ...... 55
Data Collection ...... 55
Data Analysis...... 55
Chapter Summary ...... 56
FINDINGS ...... 57
Mean Scores...... 59
x Self Directed Learning Readiness Scale/Learning Preference Assessment (SDLRS/LPA) ...... 59
National Board of Medical Examiners Family Medicine Shelf Exam (NBME-FM)...... 60
Objective Structured Clinical Examination (OSCE) ...... 61
Preceptor Rating...... 62
Final Grade...... 64
Research Questions...... 65
Research Question 1 ...... 65
Research Question 2 ...... 67
Research Question 3 ...... 68
Research Question 4 ...... 69
Research Question 5 ...... 69
Chapter Summary ...... 72
CONCLUSIONS, DISCUSSION AND RECOMMENDATIONS...... 73
Conclusions and Discussion ...... 74
Levels of Self-Directed Learning Readiness of Medical Students ...... 75
SDLRS/LPA Scores and NBME-FM ...... 77
SDLRS/LPA Scores and OSCE ...... 77
SDLRS/LPA Scores and Preceptor Ratings...... 78
SDLRS/LPA Scores and Final Grade ...... 79
Further Discussion ...... 79
Limitations ...... 80
xi Recommendations for Future Study ...... 81
Summary ...... 84
APPENDIXES
A Final Grade Formula…………………………...... …………………………....87
B Objective Structured Clinical Examination (OSCE) Form...... 90
C Preceptor Rating Form...... 94
D Self-Directed Learning Readiness Scale / Learning Preference Assessment ..... 99
E How to Interpret Your SDLRS/LPA Score...... 105
REFERENCES……………………………………………………………………….. 107
xii
LIST OF TABLES
Table 1. The National Board of Medical Examiners Family Medicine Shelf Exam (NBME-FM) Content ...... 29
Table 2. Total Graduates at UTMB by Race/Ethnicity within Gender, 2006 ...... 49
Table 3. Mean SDLRS Scores for Third-Year Medical Students and Previously Reported Samples...... 66
Table 4. Number of Third-Year Medical Students by SDLRS/LPA Category ...... 67
Table 5. Predictor Variables for Self-Directed Learning Readiness Scale/ Learner Preference Assessment (SDLRS/LPA) ...... 70
Table 6. Predictor Variables for National Board of Medical Examiners Family Medicine Shelf Exam (NBME-FM) ...... 71
xiii
LIST OF FIGURES
Figure 1. The problem-based learning cycle………….………………………………….23
Figure 2. Mean Self-Directed Learning Readiness Scale/Learner Preference Assessment(SDLRS/LPA) frequency…………….…………………..………..59
Figure 3. Mean National Board of Medical Examiners Family Medicine Shelf Exam(NBME-FM)scores.…………………………………………………..…60
Figure 4. Mean Objective Structured Clinical Examination (OSCE) scores…………….62
Figure 5. Mean preceptor rating scores…………………………………………………..64
Figure 6. Mean final grade scores……...…………...……………………………………65
xiv
CHAPTER 1
INTRODUCTION
Perhaps no other profession has more impact on our society’s health than the
medical profession. The assistance of physicians in the prevention and treatment of
disease is paramount to our quality of life. Physicians provide essential services to society
that have profound and far-reaching effects on lives.
For those who aspire to become physicians, medical school is a demanding, time-
consuming, and costly endeavor. Typically, medical students undertake four years of
undergraduate education, four years of medical school and three to eight years of
specialty-specific internships and residency (U.S. Department of Labor, Bureau of Labor
Statistics, 2007). The American Association of Medical Colleges (AAMC) (2007)
reported that the mean for tuition, fees, and health insurance for the first year alone for students in public medical schools was $20,978 for state residents and $39,974 for non- residents. Trends in these costs have resulted in medical educational debt that is
considerably higher today than just a few short years ago (Fang, 2004). At the University of Texas Medical Branch School of Medicine (UTMB), the proposed resident tuition and fees for the first year of medical school are approximately $11,795, while non-residents pay approximately $24,895 (UTMB School of Medicine Bulletin, 2007-2009), It is apparent that selecting and retaining students who have the greatest chance for success in medical school is of great importance.
1 In the past, medical students have exhibited an exceptionally high graduation rate
(96% completed within a 10-year period) with only a 4% attrition rate (Garrison,
Mikesell, & Matthew, 2007). This is even more remarkable when compared to the 62%
completion rate and 38% attrition rate of all others in postgraduate education. Despite the
high graduation rate of medical students, some studies have predicted physician shortages
in the United States (Garrison, Mikesell, et al.).
In 2006, there were 69,167 students enrolled in medical schools in the United
States (AAMC, 2007). In June 1996, the AAMC, in response to evidence that the United
States will confront a physician shortage, called for an increase in enrollment of 30% by
the year 2017 (Garrison, Matthew, & Jones, 2007). There has been a 5-year growth trend of first-time applicants, and the future applicant pool is projected to be deep enough to substantiate this 30% increase (Garrison, Matthew, et al.). In fact, the U.S. Department of
Labor, Bureau of Labor Statistics (2007), projects that employment for physicians will grow faster than average for all occupations through the year 2014. With this growth, predicting success in medical school by measuring and evaluating the behaviors and traits necessary for success may become even more vital (Garrison, Matthew, et al.).
The contemporary practice of medicine requires self-efficacy, experience, and the rigorous application of critical thinking skills (Yalcin, Karahan, Karadenizil, & Sahin,
2006). In fact, the Watson-Glaser Critical Thinking Skills Appraisal has been shown to be a moderate predictor of success in preclinical medical education (Scott & Markert,
1994). Unfortunately, some studies suggest that physicians may not be adept in accurate self-assessment and may be lacking in meta-cognitive abilities (Violato & Lockyer,
2006). Vigorous shifts in the health care system, the proliferation of vital, current
2 information, rapid expansion of knowledge, and the intricacies of reflective practice
necessitate that those who practice have the ability to be self-directed learners in order to
grow as professionals and provide the highest quality of patient care (Ainoda, Onishi, &
Yasuda, 2005; Williams, 2004). Constructivist-oriented and based on social-cognitive
theory, self-directed learning (SDL) promotes self-efficacy, a necessary attribute in
today’s complex information world (Bradley, Oterholt, Nordheim, & Bjørndal, 2005;
Bulik, Burdine, & Shokar, 2007). Moreover, this capacity to identify learning needs and direct and regulate the learning experience in a linear and non-linear fashion is highly associated with success as a physician, which may translate directly to patients’ health
(Bulik, 2003; Yalcin et al., 2006).
Recognizing these facts, the Accreditation Council for Graduate Medical
Education (ACGME) has established implementing problem-based learning (PBL) in medical school as one of its six core competencies. Two of the key components of PBL are directly related to self-directed learning--lifelong learning and self-reflection
(ACGME, 2006).
In 1993, the UK Medical Council published Tomorrow’s Doctors, which called for curtailing the factual content and increasing the SDL capacity in medical education
(Whittle & Murdoch-Eaton, 2004). In response to this need for physicians to be self- directed, some medical schools, such as the University of Texas Medical Branch at
Galveston (UTMB), have adopted an Integrated Medical Curriculum (IMC). In the first two years, a physician is assigned to facilitate a group of 6-8 students with a PBL strategy where “all courses are interdisciplinary and are based on self-directed, problem-based learning, with supplemental large-group lectures and laboratory sessions” (UTMB School
3 of Medicine Bulletin, 2007-2009, p. 14). While the medical profession has demonstrated commitment to developing physicians who are self-directed, continuing learners, many medical preparation programs are still struggling with the implementation of this concept.
Rationale of the Study
Problem-based learning puts more emphasis on real-world clinical performance rather than rote memorization. Sloan, Donnelly, Schwartz, and Strodel (1995) declare,
“The primary goal of training programs is to produce competent practitioners. Physicians recognize that clinical competence is determined by more than knowledge. Although a sound knowledge base is vital, clinical competence encompasses numerous other domains” (p. 736). As such, much of present-day medical training now focuses on clinical competence outcomes based on observable behaviors (Carraccio & Englander,
2000).
Given the facts that medical school acceptance is highly competitive and the rigors demanding, examinations used to predict success are essential. The Medical
College Admissions Test (MCAT) (2007) is a knowledge-based, multiple-choice examination used by most medical schools to determine an examinee’s aptitude for the rigors of medical education. The MCAT has been shown to have small to medium predictive validity for both medical school performance and United States Medical
Licensing Examinations (USMLE) in a recent meta-analysis (Donnon, Paolucci, &
Violato, 2007). While the MCAT is not the lone factor in selecting applicants, experts have called for additional screening and selection tools to enhance the probability of success. Medical schools also realize that knowledge-based exam scores are only part of the process. In 2001, the AAMC conducted a cross-validation study to identify relevant
4 academic and non-academic attributes of medical students and residents (Etienne &
Julian, 2001). The researchers found that these behaviors that were demonstrated through
critical incident reports were: shaping the learning process (“taking an active role in their
own learning and knowledge acquisition,” para. 6), self-management and coping skills
(“balancing the demands of medical school with other aspects of life by prioritizing,
setting time limits, adapting to diverse environments, and appropriately requesting
feedback and assistance from professors or other students,” para. 6), fostering a team
environment, interpersonal skills and professionalism, interacting with patients and
families, technical knowledge and skill, ethical behavior, mentoring and educating
students, and maintaining calm under pressure (Etienne & Julian).
Consequently, further investigation of test instruments that measure qualities and
traits defined as being essential to practice should be examined to assess their predictive
value. Additionally, curriculum changes explicitly designed to enhance SDL should be
investigated to determine if the desired effects are being realized.
The National Board of Medical Examiners provides subject tests in the basic and
clinical sciences for the purpose of assessing educational achievement (National Board of
Medical Examiners, 2008). That organization’s Family Medicine Shelf Examination
(NBME-FM) is typically taken during the third year of medical school.
Problem Statement
Previously, a study done with third-year medical students enrolled in an integrated medical curriculum (IMC) have shown significantly (p < .01) higher mean scores (236.6) when compared to the general population (214.0) on Guglielmino’s (1978) Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) (Bulik, 2003).
5 Quantitative research has also suggested that medical students educated in a PBL
curriculum demonstrated superior clinical reasoning skills, time management skills and
clinical information retention (Schwartz, Burgett, Blue, Donnelly, & Sloan, 1997;
Schwartz, Donnelly, Nash, & Young, 1992).
UTMB at Galveston has adopted an IMC that provides PBL/Evidence-based
medicine (EBM) as the main mode to accomplish curricular goals (Bulik, 2003). Students
form 6-8 member problem-based learning groups and present patient case studies to a
preceptor the first two years of the program and participate in clinical rotations with
integrated web-based studies during the third and fourth years leading up to their
residency (Bulik et al., 2007).
Partially due to the fact that the desirable traits of physicians are fostered by SDL,
PBL/EBM is becoming more common in medical school curricula. Therefore, the
contribution of SDL throughout the medical school experience is worthy of investigation.
Purpose of the Study
The purpose of this study was to investigate (a) the self-directed learning (SDL)
readiness of third-year medical students in comparison to previously reported scores for
the general population and (b) the relationship between SDL readiness and knowledge-
based and performance-based measures of success in a medical school using an
integrated medical curriculum; and (c) to determine if knowledge-based and
performance-based measures of success are significant in predicting SDLRS/LPA and
National Board of Medical Examiners Family Medicine Shelf Examination (NBME-FM) scores.
6 Previous studies have used either knowledge-based or performance-based test scores in their assessment of medical school success. This investigation will use knowledge-based test scores, performance-based test scores, and a composite of knowledge-based and performance-based test scores as measures of success.
Research Questions
1. Are third-year medical students’ scores on the Self-Directed Learning Readiness
Scale/Learner Preference Assessment (SDLRS/LPA) higher than the scores of the
general adult population reported by Guglielmino & Guglielmino in 1988 (214.0
+ 25.59)?
2. Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores
and knowledge-based measures of success (National Board of Medical
Examiners’ Family Medicine Shelf Examination [NBME-FM] scores)?
3. Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores
and performance-based measures of success (Objective Structured Clinical
Examination [OSCE] scores, preceptor rating scores)?
4. Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores
and the combination of knowledge-based and performance-based measures of
success (final grade)?
5. Are knowledge-based and performance-based measures of success (Objective
Structured Clinical Examination scores, preceptor ratings scores, final grade)
7 significant in predicting Self-Directed Learning Readiness Scale/Learner
Preference Assessment (SDLRS/LPA) scores and/or National Board of Medical
Examiners Family Medicine Shelf Examination (NBME-FM) scores?
Hypotheses
1. Scores of third-year medical students’ Self-Directed Learning Readiness
Scale/Learner Preference Assessment (SDLRS/LPA) scores are not higher than the scores of the general adult population reported by Guglielmino & Guglielmino in
1988 (214.0 + 25.59).
2. There is no significant correlation between third-year medical students’ Self-
Directed Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA)
scores and knowledge-based measures of success (National Board of Medical
Examiners Family Medicine Shelf Examination [NBME-FM] scores).
3. There is no significant correlation between third-year medical students’ Self-
Directed Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA)
scores and performance-based measures of success (Objective Structured Clinical
Examination [OSCE] scores, preceptor rating scores).
4. There is no significant correlation between third-year medical students’ Self-
Directed Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA)
scores and the combination of knowledge-based and performance-based measures
of success (final grade).
5. Knowledge-based and performance-based measures of success (Objective
Structured Clinical Examination scores, preceptor ratings scores, final grade) are
not significant in predicting Self-Directed Learning Readiness Scale/Learner
8 Preference Assessment (SDLRS/LPA) scores and/or National Board of Medical
Examiners Family Medicine Shelf Examination (NBME-FM) scores.
Significance of the Problem
With medical school entrance becoming increasingly competitive, benchmarks to
predict candidates’ success and non-success are needed. Standard benchmarks such as
undergraduate grade point average and MCAT scores are proficient in predicting success
in NBME-FM performance, but not useful in predicting the transferable skill of clinical
performance (Silver & Hodgson, 1997). While no one criterion can be used to qualify the
predicted success of any student, it is helpful to measure students in areas that have been
demonstrated to be essential in determining success. Being self-directed has been shown
to be a contributing attribute in this profession and, therefore, one that requires
investigation. Further, it has been shown that medical school students enter their
programs with disparate degrees of SDL readiness. One of the major tenets of medical
education is to cultivate competencies, such as SDL, that will transfer into lifelong
independent learning (Whittle & Murdoch-Eaton, 2001). Accordingly, Harvey, Rothman,
& Fecker (2003) state that, “becoming an independent and self-directed lifelong learner is
one of the critical outcomes of undergraduate medical education” (p. 1259).
In addition, if success in third-year practical application courses is greater in those who possess higher levels of SDL, those responsible for planning, organizing and delivering IMC should investigate opportunities to prepare students to increase SDL skills. The results of this study could contribute to the research base undergirding the efforts of physician preparation programs to increase the clinical effectiveness of their graduates.
9 Definition of Terms
Accreditation Council for Graduate Medical Education (ACGME). “The
Accreditation Council for Graduate Medical Education is responsible for the post-MD
medical training programs within the United States. Accreditation is accomplished
through a peer-review process and is based upon established standards and guidelines”
(ACGME, 2006, para. 1).
American Association of Medical Colleges (AAMC). The American Association of Medical Colleges is a nonprofit association of medical schools, teaching hospitals, and academic societies. It is “committed to improving the nation’s health through medical education, research, and high quality patient care” (AAMC, 2005, para. 1).
American Board of Medical Specialists (ABMS). The American Board of Medical
Specialists was established to assure competent board certified physicians. Today, over
85% of physicians practicing in the United States have been certified by the ABMS
(Miller, 2005).
American Board of Surgery In-Training Examination (ABSITE). Surgical residents take the ABSITE to assess clinical and basic science knowledge. This objective test establishes whether the resident may progress in the program of study (ABSITE,
2008).
Andragogy. Andragogy is a model of adult learning developed by Malcolm
Knowles in the 1960’s. It is defined as “the art and science of helping adults learn”
(Knowles, 1968, p. 351), and distinguishes itself from pedagogy by the expectation of increased learner responsibility and self-direction in learning.
10 Constructivism. Constructivism is a theory that asserts learning is a process of constructing meaning from individual experiences. (Merriam & Caffarella, 1999). It is an active process in which learners construct or reconstruct knowledge networks (Dolmans,
De Grave, Wolfhagen, & van der Vleuten, 2005). Learning from a constructivist point of view is reliant on having previous experiences, introducing conflict, negotiating, and developing new schemes of knowledge that adapt to the experience (Merriam &
Caffarella). According to Candy (1991), “the constructivist view of learning theory is particularly compatible with the notion of self-direction, since it emphasizes the combined characteristics of inquiry, independence, and individuality in learning a new task” (p. 278).
Cooperative learning. Cooperative learning is defined as having the “presence of joint goals, mutual rewards, shared resources, and complementary roles among members of a learning group” (Qin, Johnson, & Johnson, 1995, p. 131).
Evidence-based medicine (EBM). EBM is defined as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” (Sackett, Rosenberg, Gray, Haynes & Richardson, 1996, p. 71).
Final grade scores. In the institution studied, a formula equation is used to take into account variability in faculty observation (OSCE AVG) and evaluation (preceptor
AVG). The NBME-FM is also equated in the formula to determine the final grade.
Therefore, it is a reflection of the combination of standardized and practical-based measures (Appendix A).
Humanistic philosophy. “Humanist theories consider learning from the perspective of human potential for growth. The study of the affective as well as cognitive
11 dimensions of learning was informed in part by Freud’s psychoanalytical approach to
human behavior” (Merriam & Caffarella, 1999, p. 256).
Information processing theory. This theory involves activating prior knowledge,
encoding specificity, and knowledge elaboration (Albanese, 2000; Schmidt, Vermeulen,
& Van Der Molen, 2006).
Integrated medical curriculum (IMC). The integrated medical curriculum
examined in this study is a curriculum employed at UTMB where students form 6-8-
member problem-based learning groups and present patient case studies to a preceptor the
first two years of the program and participate in clinical rotations with integrated web-
based studies during the third and fourth years leading up to their residency (Bulik et al.,
2007). The IMC presents the integration of PBL/EBM strategies which promote SDL in
students.
Lifelong learning. “In a world of accelerating change, learning must be a lifelong
process. Therefore, schooling must be concerned primarily with developing the skills of
inquiry and adult education must be primarily concerned with providing resources for
self-directed inquirers” (Knowles, 1970, p. 19).
Medical College Admissions Test (MCAT). The MCAT is a multiple-choice, standardized test measuring “problem solving, critical thinking, writing skills, and knowledge of science concepts and principles prerequisite to the study of medicine”
(MCAT Web Site, 2007, para. 2). This examination is used by most medical schools to determine an examinee’s aptitude for the rigors of medical education. It provides scores in verbal reasoning, physical sciences, writing sample and biological science (MCAT
Web Site, 2007).
12 National Board of Medical Examiners (NBME). The NBME provides subject
tests in the basic and clinical sciences for the purpose of assessing educational
achievement (NBME, 2008). The National Board of Medical Examiners Family
Medicine Shelf Exam (NBME-FM) used in this study consists of 100 questions, most of
which are single-best answer (a through e) based on lengthy patient-based scenarios. The
NBME-FM is timed at 2 hours, 10 minutes and focuses on “normal growth and
development and general principles of care throughout the lifespan,” (Southwestern
Medical Center, 2008, Examination Focus section, para. 1) and “diagnosis and initial management of common life-threatening and debilitating diseases” (Southwestern
Medical Center, Examination Focus section, para. 1).
Objective Structured Clinical Medical Examination (OSCE). The Objective
Structured Clinical Medical Examination is “an assessment tool in which the components of clinical competence such as history taking, physical examination, simple procedures, interpretation of lab results, patient management problems, communication, attitude etc. are tested using agreed checklists and rotating the student around a number of stations”
(Ananthakrishnan, 1993, p. 82). OSCEs are evaluated at UTMB with a clinical competence examination form (Appendix B).
Preceptor ratings. Students are assigned between one and three faculty preceptors who complete assessments for their one-month clinical experience. Although students are assigned one primary preceptor, if a student works with multiple faculty, up to three may be assigned. Students are assessed in history-taking skills, physical examination skills, communication skills, problem-solving skills and professionalism. Preceptor Rating
13 scores are determined by using a clerkship evaluation form, a 10-item standard evaluation
form used to rate student performance (Appendix C).
Problem-based learning (PBL). Problem-based learning has been defined as “an
instructional strategy in which students identify issues raised by specific problems to help
develop an understanding of underlying concepts and principles” (Norman & Schmidt,
1992, p. 557).
Self-determination theory. This theory differentiates autonomous learning from
controlled learning. Since autonomous learning centers around the learner’s volition,
greater opportunities exist for conceptual understanding, academic achievement,
creativity, persistence, and psychological utility (Williams, Saizow, & Ryan, 1999).
Self-directed learning (SDL) Self-directed learning “describes a process in which
individuals take the initiative, with or without the help of others, in diagnosing their
learning needs, formulating learning goals, identifying human and material resources for
learning, choosing and implementing appropriate learning strategies, and evaluating
learning outcomes” (Knowles, 1975, p. 18).
Self-Directed Learning Readiness Scale/Learner Preference Assessment
(SDLRS/LPA). The SDLRS/LPA (Appendix D) is the most commonly used valid
quantitative tool to measure an individual’s existing readiness for managing his or her
own learning (Merriam, Caffarella, & Baumgartner, 2007). How to interpret the
SDLRS/LPA is found in Appendix E.
Self-efficacy: “Self-efficacy is the belief in one’s capabilities to organize and execute the sources of action required to manage prospective situations” (Bandura, 1986,
14 p. 391). A person’s belief in his or her own efficacy is the main determinant of the choices that are made in accomplishing a desired task (Derrick, 2003).
Social-cognitive theory: Bandura (1989) describes social cognitive theory as “a model of emergent interactive agency” (p. 1175). This theory postulates that “any account of the determinants of human action must, therefore, include self-generated influences as a contributing factor” (Bandura, p.1175).
United States Medical Licensing Examination (USMLE, 2007). The United States
Medical Licensing Examination is three-step multiple-choice test used for medical licensure.
1. Step 1 covers basic science as it relates to individual organ systems (USMLE).
2. Step 2 scores “assess whether medical school students or graduates can apply
medical knowledge, skills and understanding of clinical science essential for
provision of patient care under supervision” (USMLE, Licensing Exams section,
para.3).
3. Step 3 scores “assess whether medical school graduates can apply medical
knowledge and understanding or biomedical and clinical science essential for the
unsupervised practice of medicine” (USMLE, Licensing Exams section, para. 4).
Watson-Glaser Critical Thinking Appraisal (WGCTA). The Watson-Glaser Critical
Thinking Appraisal is an 80-item inventory designed to assess critical thinking ability, which is the composite of attitudes, knowledge, and skills. It is divided into five subscales: inference, recognition of assumptions, deduction, interpretation, and evaluation of arguments. This appraisal has been shown to be a moderate predictor of success in preclinical medical education (Scott & Markert, 1994).
15 Limitations and Delimitations
Limitations
There is a potential attenuation of the correlation between NBME-FM and performance-based measures due to the restricted variance of the NBME-FM.
Additionally, the SDLRS/LPA scores may be truncated as well. It must also be assumed that all students answered the SDLRS/LPA truthfully and that variables not investigated were equal across the sample.
Another limitation is that there may be undue bias in the student population towards preferring to learn in a self-directed manner. White (2007) has shown that some medical students choose PBL curricula based on their individual learning preference.
Therefore, this preference may differentiate characteristics in these students from other medical school populations.
Several disadvantages of PBL/EBM have been identified. They include an increase in instructor/staff time, preparation/planning (Tärnvik, 2007; Yalcin et al., 2006) maintenance costs and poor course organization (Yalcin et al.). Most evident, however, is the increased content anxiety inherent in this method of learning (Williams, 2004; Yalcin et al.), which may lead to poor attendance rates and difficulty adapting to the approach
(Bradley et al., 2005). Others argue that the intrinsic motivation of medical students is such that any method of instruction will produce desirable outcomes (Bradley et al.).
Delimitations
The scope of this study was bound to a single large university medical school and the results may not be generalizeable to other university medical schools, whose demographics and curricula may differ. The medical school was selected due to data
16 availability. Further qualitative analysis of disparate medical schools would be beneficial in order to generalize the data.
Organization of the Study
This study examined the relationship of SDL readiness in third-year medical school students to predictors of knowledge-based success (NBME-FM), performance- based success (OSCE, preceptor rating) and the combination of knowledge-based and performance-based success (final grade) in medical school. This study is structured into five chapters. The first chapter includes the introduction, rationale of the study, problem statement, purpose of the study, research questions, hypotheses, and significance of the problem, a definition of terms, and limitations and delimitations. The second chapter presents a literature review related to physicians and physician preparation, integrated medical curriculum including problem-based learning/evidence–based medicine, the
NBME-FM, the OSCE, and self-directed learning, particularly pertaining to the medical field. Chapter 3 outlines the methodology of the study, including subjects, institutional setting, instrumentation, procedures, and methods for data collection and analysis.
Chapter 4 presents the quantitative findings using mean score comparisons, correlation analysis, and regression analysis specific to the research questions. The results of the hypotheses testing will also be presented. Chapter 5 provides a discussion regarding the study summary, conclusions, significance of the investigation and recommendations for future research.
17
CHAPTER 2
LITERATURE REVIEW
A review of the literature pertaining to physicians and physician preparation
reveals that the curricula chosen by medical schools greatly impact professional
preparation and continuous lifelong learning. A review of the integrated medical
curriculum (IMC) based on problem-based learning and evidence-based medicine is
presented. The history and practice of the National Board of Medical Examiners
assessments and Objective Structured Clinical Examination (OSCE) are discussed. The
history of self-directed learning is discussed, highlighting the major studies. Finally, the
research on medical students as self-directed learners is presented.
Overview: Physicians and Physician Preparation
There were approximately 567,000 jobs held by physicians and surgeons in the
United States in 2004 (U. S. Department of Labor, Bureau of Labor Statistics, 2007). Of
those, about 14% were self-employed and not incorporated (U. S. Department of Labor,
Bureau of Labor Statistics). Sixty percent of physicians and surgeons worked in offices of
multiple physicians, 16% worked in private hospitals, and others practiced in various
other agencies, such as hospitals, colleges and universities, outpatient care centers etc.
(U. S. Department of Labor, Bureau of Labor Statistics). The American Medical
Association reports that approximately 40% employed in patient care were in primary care, while the remaining 60% practiced in a subspecialty such as anesthesiology,
18 psychiatry, or a surgical specialty (U. S. Department of Labor, Bureau of Labor Statistics,
2007).
Physicians work long and irregular hours (U. S. Department of Labor, Bureau of
Labor Statistics, 2007). Over 33% of all full-time physicians worked 60 hours per week
or more in 2004 (U. S. Department of Labor, Bureau of Labor Statistics). According to
the Medical Group Management Association’s Physician Compensation and Production
Survey (2005), median compensation ranges from $132,953 to $321,686 depending on specialty.
In 2006, there were 69,167 students enrolled in medical schools in the United
States (AAMC, 2007). For students preparing to become physicians, the requirements are among the most demanding for all occupations. According to the U. S. Department of
Labor, Bureau of Labor Statistics (2007) most students will attend 4 years of undergraduate school, 4 years of medical school and 3 to 8 years of residency, depending on the specialty selected.
The third year of medical school is a pivotal point in the success of the student. In many cases, mental weariness and waning motivation of third-year students have resulted in significant reductions in self-study and perceived depth of discussion (Musal, Gursel,
Taskiran, Ozan, & Tuna, 2004). It is essential at this point to offer challenging curricula in which students are immersed in meaningful learning experiences.
Integrated Medical Curriculum
The curricular approaches for medical students have evolved over the past 25 years. The last decade, in particular, has witnessed reform in curriculum to increase public assurance that graduates are competent physicians (Appel, Friedman, Fazio,
19 Kimmel, & Whelan, 2002). According to the Association of American Medical Colleges
(1998) the process of improving the quality of medical education curriculum is
continuous. Medicine must always be responsive to “evolving societal needs, practice
patterns, and scientific developments” (Association of American Medical Colleges, p. 9).
While traditional classroom lectures still exist, newer instructional methods such as
computer-based video training for surgical skills are being used with much success
(Jowett, LeBlanc, Xeroulis, MacRae, & Dubrowski, 2007). While part of the shift in
mode of training may be due to technological advances, much of the impetus for these
changes is due to increased knowledge of how learning best occurs. Lujan and DiCarlo
(2006) note that studies show that the quantity of undergraduate science education has no
effect on medical students’ academic success. However, the variance in these studies may
make it problematic to accurately assess the results. Low retention rates of basic science,
anatomy and biochemistry information prior to medical school graduation have led those
involved in curricular decision-making to rethink their strategies (Lujan & DiCarlo). As a
consequence, the philosophy that “teachers should reduce the total amount of factual
information students are expected to memorize, reduce our use of the passive lecture
format, and devote much more effort to helping students become active, independent
learners and problem solvers” (Lujan & DiCarlo, p. 17), is now prevalent in medical education. To this end, Whittle and Murdoch-Eaton (2001) state, “This change in teaching and learning methods will place a greater responsibility on individual students to manage their own learning, and highlights the need for students to develop a good standard of transferable skills” (p. 148).
20 Problem-Based Learning/Evidence-Based Medicine (PBL/EBM)
Lujan and DiCarlo (2006) state, “Lecture merely exposes students to content, and
exposure is not sufficient for learning. Active processing of information, not passive
reception of information leads to learning” (p. 17). Research suggests that learners can be
broadly categorized into two groups: internally-motivated self-directed learners and
externally motivated teacher-centered learners. Medical schools have increasingly
restructured their approach to contemporary medical education to produce the self-
motivated learner (Bradley et al., 2005)
Two closely related methods identified to facilitate the development of self-
directed learning are problem-based learning (PBL), and evidence-based medicine
(EBM) curricula (Williams, 2004). These forms of self-instruction have increased
substantially over the past 15 years. In fact, 79% of medical schools reported using self-
instruction in their curricula in 1995-96. This statistic jumped to 94% in 1998-99
(Albanese, 2000). In fact, the enthusiasm for this type of learning has spawned the Center
for Problem-Based Learning at Southern Illinois University and the Problem-Based
Learning Assessment and Research Center at the University of Newcastle in Australia
(Albanese).
Problem-based learning, which was introduced in the late 1970’s (Tärnvik, 2007)
by Barrows and Tamblyn at McMaster University in Canada (Trevena, 2007), has been
defined as “an instructional strategy in which students identify issues raised by specific
problems to help develop an understanding of underlying concepts and principles”
(Norman & Schmidt, 1992, p. 557). The objectives of this learning strategy are vital
21 knowledge acquisition, transfer of knowledge in clinical settings, and SDL (Musal et al.,
2004).
Albanese (2000) describes PBL with relation to autonomous (self-directed) learning:
A learning climate which promotes autonomous motivators includes one in which
educators take the perspective of the students into account, provide relevant
information and opportunities for choice, and to encourage students to accept
more responsibility for their own learning and behavior. It also includes teachers
being meaningfully involved in students’ learning through dialogue, listening,
asking students what they want, providing factual information and advice and
suspending judgment when soliciting the opinions and reactions of the students.
Such an environment minimizes control and pressure while encouraging a high
level of performance. Autonomous motivators would be especially compatible
with the collaborative learning environment. Further, PBL would seem to be an
easier fit with autonomous motivators than would the traditional curriculum. (p.
734)
Grounded in cognitive psychology, PBL instructors present to small groups scenarios or phenomena that require further clarification (Tärnvik, 2007). At the core of
PBL is the opportunity for students to learn contextually. Researchers have argued that learning in the context of how the information will be used translates into students being better able to apply the information than they would be if they had learned it in a competitively-graded, lecture-based environment. Contextual learning is paramount in
22 transferability of medical skills (Albanese, 2000). Figure 1 graphically represents the process of PBL in relation to SDL.
Figure 1. The problem-based learning cycle.
Note: Reprinted with permission from Springer Science + Business Media: Educational Psychology Review, “Problem-based learning: What and how do students learn?” 2004, 16(3), p. 237, by Hmelo-Silver, C.E., Figure 1, © Springer Netherlands.
23 One of the basic tenets of PBL is the effective use of cooperative learning (CL)
relevant to the study domain (Schmidt et al., 2006). Cooperative learning is defined as
“the presence of joint goals, mutual rewards, shared resources and complementary goals among members of the group (Qin et. al., 1995, p. 131).” Results of Qin et al.’s meta- analysis revealed that CL resulted in higher quality problem solving than competition.
The major advantages of CL in relation to problem-solving were the group’s exchange of ideas and the identification of individual errors (Qin et al.). In fact, research suggests that trained peer-facilitated PBL groups are as effective as trained faculty-facilitated groups in
OSCE performance (Steele, Medder, & Turner, 2000).
While PBL has detractors (Colliver, 2000) who claim all medical education is clinically contextual, PBL is based on information-processing theory which involves activating prior knowledge, encoding specificity, and knowledge elaboration (Albanese,
2000; Schmidt et al., 2006). In 1998, Champagain et al., conducted a research study with
26 medical students engaged in a PBL curriculum. Even though the sample size was small and caution should be used in interpreting the results, they perceived that PBL facilitated the development of SDL.
Some studies suggest that students favor a combination of didactic lectures with
PBL learning strategies and they perceive this combination leads to greater SDL (Ghosh,
2007). The ACGME has recognized PBL implementation and improvement as one of their six core competencies (ACGME, 2006; Leach, 2006).
Evidence-based medicine, whose origins date back to mid-19th century Paris, is
defined as “the conscientious, explicit, and judicious use of current best evidence in
making decisions about the care of individual patients (Sackett et al., 1996, p.71).”
24 Bradley et al. (2005) identify five steps to the EBM process: “formulating clinically
important questions, efficient gathering of clinical evidence (research), critical appraisal
(assessment) of evidence, applying evidence to practice, evaluating own practice” (p.
150). As such, medical school students are commonly presented case studies that require
them to analyze the case and develop suitable questions; locate, synthesize and critically
interpret information; and apply the findings in order to best treat the patient (Bradley et
al.; Yalcin et al., 2006). In this philosophy, skilled practitioners (and those training to be)
are challenged to apply their own clinical knowledge with the best existing external
evidence in order to more effectively treat patients (Sackett et al., 1996).
Based on available research, the cognitive advantages of PBL/EBM are an increase in honing SDL skills that transfer to professional practice, motivation (Schmidt et al., 2006), scientific thinking, promoting deeper understanding (Yalcin et al., 2006), better knowledge acquisition (Bradley et al., 2005; Schmidt et al.), critical appraisal skills, learner autonomy (Bradley et al.), and problem solving (Schmidt et al.; Yalcin et al.). Candy (1991) even reports that students exposed to learning in this fashion use library resources at a greater rate than those in traditional programs. Increasing the breadth of resources is in keeping with the goals of SDL (Candy).
The interpersonal advantages of PBL/EBM are improved professional collaboration, (Schmidt et al., 2006; Yalcin et al., 2006), conflict resolution (Yalcin et al.), retention, better patient communication, teamwork, expertise in running meetings, helping colleagues’ confidence, ability to work and plan efficiently (Schmidt et al.) and improved attitudes (Bradley et al., 2005). Further, PBL enhances enjoyment of school by both student and instructor when compared to traditional curriculum (Albanese, 2000).
25 Additionally, students and faculty that attend medical schools that employ these
strategies have not only been found to like the experience more than their traditional
counterparts; they affiliate better, as well (Abraham, Upadbya, & Ramnarayan, 2005;
Albanese, 2000). In fact, PBL graduates were more likely to, “spend more time in direct
patient care, bill for more psychotherapy services per month, have an academic
appointment, enter family medicine and be in group practice” (Albanese, p. 736).
Another factor to consider is that PBL helps to develop intrinsic motivation and appears
to enhance skills in SDL (Norman & Schmidt, 1992). Some medical students have begun to use PBL/EBM curricula in the selection of which school they attend. White (2007)
studied students from traditional programs and those who employed PBL and found that
none of the students from traditional learning programs chose their program based on its
approach to learning. However, some of the students in the PBL program specifically
chose their school because of the approach (White).
Based on this model of inquiry, continuing medical education has often adopted
PBL/EBM strategies (Hiramanek, 2005). Yalcin et al. (2006) have identified that there is
some evidence that physician knowledge, skills, and abilities have increased through the
use of that method, which has directly improved patient health.
The National Board of Medical Examiners Assessments
Appel et al. (2002) report that, “knowledge of a specific content area has been
shown to be the best predictor of success in that content area” (para. 5). Knowledge-
based multiple choice testing has been a tenet of medical school evaluation since its
inception, showing high objectivity, reliability and content validity (Appel et al.). The
26 NBME is “an independent, not-for-profit organization that provides high-quality
examinations for the health professions” (NBME, 2008, p. 1). Its mission is:
to protect the health of the public through state of the art assessment of health
professionals. While centered on assessment of physicians, this mission
encompasses the spectrum of health professionals along the continuum of
education, training and practice that includes research in evaluation as well as
development of assessment instruments. (NBME, 2008, p. 1)
Founded in 1915, the organization has set the standard by which medical
licensure can be judged (NBME, 2008). The organization provides subject tests in the
basic and clinical sciences for the purpose of assessing educational achievement. The
NBME I relies on a student’s ability to recall facts related to the discipline, while the
NBME III requires the synthesis and analysis of facts and application to patient cases. “In the short-run the more teacher-centered and structured conventional curriculum better prepared the students for the NBME I, while in the long run, the more student-centered, problem-based curriculum better prepared the students for the NBME III” (Mennin,
Friedman, Skipper, Kalishman, & Snyder, 1993, p. 616).
The NBME also provides a number of subject area examinations; the Family
Medicine area examination was used in this study (NBME-FM). The examination consists of 100 items; and 2 hours, 10 minutes is allotted. The focus of the examination is “normal growth and development and general principles of care throughout the life span; and diagnosis and initial management of common life-threatening and debilitating diseases”
(Southwestern Medical Center, 2008, para. 1). A good portion of the examination is based on extensive patient-based scenarios where the patient medical history and physical
27 findings are provided. The exam consists of identifying the single best answer of five
choices and correctly identifying extended matching items (Southwestern Medical
Center). Table 1 lists the content of the Family Medicine Shelf Exam with the percentages
of questions addressing each content area.
History of the Objective Structured Clinical Examination (OSCE)
Knowledge-based examinations have limits and they often fail to measure qualities in a
physician that patients value the most (humanistic behavior, real-world performance, and
the ability to learn when not under examination scenarios) (Dornan et al., 2004). The
OSCE was developed by Harden, Stevenson, Downie and Wilson (1975) at the
University of Dundee in Scotland to allow for a more complete evaluation of student clinical performance. The OSCE consists of students rotating through a number of stations within a set time period while faculty members evaluate their skills utilizing checklists (Harden et al.). Students are placed in clinically relevant simulations where they are directly observed and assessed (Karani, Leipzig, Callahan, & Thomas, 2004).
Therefore, the examination is viewed as, “a measure of clinical competence that focuses on outcomes via observable behaviors” (Carraccio & Englander, 2000, p. 736). The
OSCE is determined most often as a result of blueprinting the course objectives into the stations to assure learning outcomes and objectives are met (Malloy, Perkowski,
Callaway, & Speer, 1998).
28 Table 1 The National Board of Medical Examiners Family Medicine Shelf Exam (NBME-FM) Content
______Content % of Questions ______Distribution Across Age Groups • Childhood 5-15 • Adolescence 5-10 • Adulthood 65-75 • Geriatric 5-15 General Principles 1-5 • Infancy and Childhood (normal growth and development) • Adolescence (sexuality, separation from parents/autonomy, puberty) • Senescence (normal physical and mental changes associated with aging) • Medical Ethics and Jurisprudence Organ Systems • Immunological Disorders 5-10 • Diseases of the Blood and Blood-Forming Organs 5-10 • Mental Disorders 5-10 • Diseases of the Nervous System and Special Senses 5-10 • Cardiovascular Disorders 10-15 • Diseases of the Respiratory System 10-15 • Nutritional and Digestive Disorders 10-15 • Gynecological Disorders 5-10 • Renal, Urinary, and Male Reproductive System 5-10 • Disorders of Pregnancy, Childbirth, and the Puerperium 1-5 • Disorders of the Skin and Subcutaneous Tissues 1-5 • Diseases of the Musculoskeletal System and Connective Tissue 5-10 • Endocrine and Metabolic Disorders 5-10 Physician Tasks • Promoting Health and Health Maintenance 15-20 • Understanding Mechanisms of Disease 20-25 • Establishing a Diagnosis 35-40 • Applying Principles of Management 20-25
Note. Source: National Board of Medical Examiners Subject Examinations: Clinical Scientific Disciplines.
29 Prior to the inclusion of the OSCE, subjective faculty evaluations and knowledge- based multiple-choice examinations were used exclusively (Harden et al., 1975).
Research has shown that subjective evaluations tend to inflate actual performance
(Goetzl, Cohen, Downing, Erat, & Jessiman, 1973; Schwartz, Donnelly, Sloan, Johnson,
& Strodel, 1995; Wray & Friedland, 1973) and have limited reliability (Ansell et al.,
1979; Matsell, Wolfish, & Hsu, 1991; Maxim & Dielman, 1987; Schwartz, Donnelly,
Drake, & Sloan, 1993; Schwartz et al., 1995). In a study with 388 medical students, results revealed that adding the OSCE to clinical examination scores awarded more students into merit and distinction categories (Konje, Abrams, & Taylor, 2001).
Additionally, knowledge-based multiple choice exams are limited to measuring knowledge base only and not addressing other competencies required of competent physicians (physical examination skills, interpersonal skills, technical skills, problem- solving abilities, patient treatment skills) (Sloan et al., 1995).
One of the advantages of using the OSCE as an assessment tool is that the criteria for grading are controlled and, therefore, the examination is easy to repeat or duplicate
(Sloan et al., 1995). Also, students are placed in specific scenarios that cannot be duplicated by paper and pencil case examinations (Karani et al., 2004; Sloan et al.).
During the course of the examination, the interpersonal skills necessary to become a competent practitioner are easily identifiable (Sloan et al.). As an ancillary benefit, the
OSCE demonstrates faculty acknowledgment of the importance of clinical skills (Sloan et al.).
However, according to Sloan et al. (1995), “perhaps the greatest benefit of the
OSCE is that it allows identification of problem areas” (p. 741). In fact, the immediate
30 feedback provided during the OSCE has been shown to both improve competency at ensuing stations and improve the quality of learning (Hodder, Rivington, Calcutt, & Hart,
1989).
The challenges or disadvantages of using the OSCE as an assessment tool are: preparation of stations (Harden et al., 1975), the cost and time to administer, depending on many factors (number of stations, number of students, honoraria to patients and faculty etc.) and taxing of institutional resources (Carraccio & Englander, 2000; Karani et al., 2004). The recruitment and consent of patients can be a time-consuming process, particularly if emergencies ensue in that population (Karani et al.). Also, security of information during the rotation process may be a concern (Rutala, Wizke, Leko, Fulginiti,
& Taylor, 1991). Student feedback is mixed on the learning experience. One study revealed students felt the OSCE was less fair and enjoyable than other forms of testing
(Lane, Ziv, & Boulet, 1999). Another study revealed students questioned whether they were examined relative to course-specific objectives (Smith, Price, & Houston, 1984).
The OSCE has been shown to be a highly reliable method of assessing clinical competence (Cohen, Reznick, Taylor, Provan, & Rothman, 1990; Matsell et al., 1991;
Petrusa, Blackwell, & Ainsworth, 1990; Sloan et al., 1995). The OSCE has also been shown to have high content validity (Cohen et al.; Hull et al., 1995; Matsell et al.; Petrusa et al; Sloan et al.), construct validity (Sloan et al.) and “provides unique information about the performance” of participants (Sloan et al., p. 735). However, validation studies have been problematic. Malloy et al. (1998) states that the criterion standard faculty ratings are often correlated to the criterion OSCE checklist, using faculty who may have knowledge and/or agreement with the underlying principle and context of the exam. This
31 may skew the correlations to be higher they actually are (Malloy et al.). Additionally, the
OSCE correlated very highly with the ABSITE scores (Sloan et al.).
Student perceptions of OSCE were examined (Pierre, Wierenga, Barton, Branday,
& Christie, 2004), and determined that most agreed they were comprehensive, covered a
wide scope of knowledge and clinical competencies, and identified weaknesses. Most
students believed the OSCE was the fairest assessment format, but many found it
intimidating and more stressful than other forms of assessment (Pierre et al.). Qualitative
data from the same study reported many positive attributes of the OSCE including that it
was comprehensive, objective, fair, and motivating (Pierre et al.).
To date, two studies have been conducted with second-year medical students to determine relationships between OSCE scores and USMLE Step 1 and Step 2 scores
(Simon, Volkan, Hamann, Duffey, & Fletcher, 2002; Simon, Bui, Day, Berti, & Volkan,
2007). With 355 second-year students as subjects, the correlation coefficient for OSCE scores in physical diagnosis with USMLE Step 1 scores was 0.41 (p < 0.001) (Simon et al., 2002). Multivariate correlations in two of seven skill areas tested (diagnosis and identification of abnormality) were significant. The predictive validity of the OSCE provided a rationale for conducting this type of examination early in a medical student’s program (Simon et al., 2002). In the second study, Simon et al., (2007) measured the relationship of an OSCE exam with USMLE Step 2 scores. There was a moderate (r =
0.385, p < 0.001) correlation between the total scores of the two tests, indicating the value of utilizing OSCEs early in students’ programs (Simon et al., 2007). The use of
OSCE as a formative and summative evaluation tool has dramatically increased in clinical and non-clinical disciplines (Pierre et al., 2004). While a paradigm shift has
32 certainly occurred in medical testing, research has indicated that several assessment methods should be used in the evaluation of medical students (Hull et al., 1995).
Self-Directed Learning (SDL)
Great thinkers spanning the recorded history of man have taken the responsibility for their own learning and thus became self-directed. The ancient Greek philosophers wrote of their capacity to function in a self-directed manner. Plato, Aristotle, and Socrates all revealed their ability to be self-directed in their learning (Kulich, 1970).
In Biblical times, there is no better example of teaching someone how to be a self- directed learner than Jesus. He often taught in parables and posed questions to his disciples and others in order for them to explore answers for themselves. “Jesus expected his students to search their minds and hearts in relation to his teaching and to consider the realities of life. In encouraging others to think for themselves, Jesus posed questions and allowed for questioning” (Pazmino, 2001, p. 73). In fact, much of the Bible offers encouragement in self-directed learning, as in Proverbs 15:14: “The discerning heart seeks knowledge… “(Holy Bible, NIV, p. 734). From a constructivist view, this certainly is in line with self-directed learning as a requisite to transformational learning (Mezirow,
2000).
Hiemstra (1999) cites Alexander the Great, Caesar, Erasmus and Descartes as highly developed self-directed learners throughout history. Colonial America was perceived as a land of opportunity, where an individual could progress based on individual initiative, ability and effort (Knowles, 1962; Lerner, 1957; McDonald, 1967).
The early settlers and colonists were reliant on self-directed learning in order to survive
(Guglielmino, Long & Hiemstra, 2004). Colonialism brought new challenges, and self-
33 directed learners such as Benjamin Franklin were vital in the development of a nation
(Brockett & Hiemstra, 1991). Not only was Franklin pivotal in framing our government,
but he is also recognized as playing a critical role in early American education as a result
of his involvement with clubs and libraries (Brockett & Hiemstra).
With the formation of cities in the Eastern United States and the Western
migration of Americans, new and disparate opportunities for self-directed learning presented themselves (Guglielmino et al., 2004). The first books dedicated to the study of
self-instruction were printed in the 1840’s: Craik’s (1840) Pursuit of Knowledge Under
Difficulties: Its Pleasures and Rewards documented the importance of education outside of schools and colleges, and Hosmer’s (1847) Self-Education: Or the Philosophy of
Mental Improvement differentiated between formal education and self-initiated education
(Guglielmino et al., 2004). In Great Britain, Smiles (1882) wrote about the virtues of personal development in Self Help. In the early 1900’s, American education reformer
John Dewey cautioned that, “the teacher should be the one who guides but does not interfere with or control the process of learning” (Williams, 2004, p. 277).
In the modern era, Cyril Houle’s (1961) landmark study, The Inquiring Mind, launched the academic study of why adults learn. Houle, founder of the first doctoral program in Adult Education at the University of Chicago, found through qualitative research that there were three distinct types of adult learners: “goal-oriented, activity- oriented, and learning-oriented” (Guglielmino et al., 2004, p. 4). He identified the last group as those engaged in, “self-directed study, in which an individual or group accepts responsibility for designing and pursuing an educative activity” (Houle, 1988, p. 92).
34 Houle’s ground-breaking study initiated interest in researchers who, to that point, considered learning in everyday life unworthy of academic study (Merriam et al., 2007).
In 1965, Johnstone and Rivera published the first major quantitative study of how adults learn. They concluded that 8% of adults surveyed between 1961 and 1962 were engaged in at least one major self-education project. They also posited that “self- instruction [was] probably the most overlooked activity in adult education” (Johnstone &
Rivera, p. 37).
Self-directed learning is a topic that has increasingly captured the attention of educators, investigators, and practitioners since the advent of Malcolm Knowles’ theory of andragogy (Guglielmino et. al., 2005). With its foundation in humanistic philosophy, andragogy posits that personal growth is the objective of learning (Caffarella & Merriam,
2000). Knowles (1970), a student of Houle’s at the University of Chicago, introduced his assumptions that adults prefer to be self-directed in their learning and followed that with a practical guide in 1975: Self-Directed Learning: A Guide for Learners and Teachers.
His theory is based on five assumptions:
1. Self-concept: As a person matures his self-concept moves from one of being
a dependent personality toward one of a being a self-directed human being.
2. Experience: As a person matures he accumulates a growing reservoir of
experience that becomes an increasing resource for learning.
3. Readiness to learn: As a person matures, his readiness to learn becomes
oriented increasingly to the developmental tasks of his social roles.
4. Orientation to learning: As a person matures his time perspective changes
from one of postponed application of knowledge to immediacy of application,
35 and accordingly his orientation toward learning shifts from one of subject-
centeredness to one of problem-centeredness.
5. Motivation to learn: As a person matures his motivation to learn is internal.
(Smith, 2002, p. 8)
The reasons Knowles (1975) gave for SDL as an important skill to possess are:
1. Individuals who take initiative in learning are more likely to retain what is
learned than the passive learner.
2. Taking initiative in learning is more in tune with our natural processes of
psychological development.
3. Many recent educational developments actually place the responsibility for
learning right on the shoulders of learners (p. 14-15).
Concurrent to Knowles’ work, another student of Houle’s, Allen Tough (1971,
1978, 1979, 1982), dissected the interviews that served as the basis of The Inquiring
Mind and developed an interview schedule designed to discern information regarding self-directed learning projects. He defined a learning project as dedicating a minimum of seven hours of deliberate learning in a 12-month period (Tough, 1971). His study and ten others conducted using the protocol he developed suggested that, “almost everyone undertakes at least one or two major learning efforts a year and some individuals undertake as many as 15 or 20. The median is eight learning projects a year” (Tough,
1979, p. 1). Tough’s interview schedule assembles information not only on the number and duration of learning projects, but also examines the reasons for learning, methods for learning, types of resources used, and who the primary planners for learning were
36 (Guglielmino et al., 2005). Knowles and Tough both stressed the self-imposed
accountability of the learner in the learning process (Guglielmino et al., 2005).
Guglielmino (1978) developed an assessment tool to determine readiness for
SDL. She selected a panel of fourteen leading authorities (including Houle, Knowles, and
Tough) and conducted a three-round Delphi survey which asked participants to list and
rate the attitudes, abilities, and characteristics of highly self-directed learners. The results
led to the formation of the Self-Directed Learning Readiness Scale/Learner Preference
Assessment (SDLRS/LPA), the most widely-used instrument in the field. Chapter 3
describes the instrument in further detail, including past studies regarding its validity and
reliability.
Some researchers (Candy, 1991; Danis & Tremblay, 1988; Mocker & Spear,
1982) criticized the process of SDL as being presented as overly linear. In 1984, Spear
and Mocker conducted qualitative research to explore triggering events, motivating
forces, how learners obtained resources, and the decision-making process of adult
learning projects. Their findings, based on 78 adults who had not earned a high school
diploma, concluded that environmental factors played a large role in self-directed
learning efforts. They found that preplanning a learning project rarely occurred in their
sample and the projects progressed, “from limited alternatives which occurred
fortuitously within [the learner’s] environment” (Spear & Mocker, 1984, p. 201).
Today, the study of SDL has broadened its appeal to include research in the
leadership genre. In Primal Leadership, Goleman, Boyatzis, and McKee (2002) state,
“The crux of leadership development that works is self-directed learning; intentionally developing or strengthening an aspect of who you are or who you want to be” (p. 109).
37 Self-directed learning has also been described as self-education, autonomous
learning, self-planned learning, adults’ learning projects, independent study,
autodidacticism, (Guglielmino et al., 2004), lifelong learning, independent study (Ainoda
et al., 2005; Guglielmino et al., 2004), self-efficacy, regulated learning, self-taught
learning (Hiemstra, 2004) continuous medical education, student-centered education
(Ainoda et al.) and a host of other terms in and out of the medical literature. The
ambiguity and interchangeable terminology has been a problem for consumers of this research. This is potentially problematic for medical educators who may conceptually embrace SDL but lack the theoretical knowledge (Ainoda et al.).
In a Medline search of scientific, research-based medical articles published from
2000-2004, a mere eight presented a concrete, precise definition of SDL (Ainoda et al.,
2005). The most frequently cited definition of SDL comes from Knowles (1975):
In its broadest meaning, “self-directed learning” describes a process in which
individuals take the initiative, with or without the help of others, in diagnosing
their learning needs, formulating learning goals, identifying human and material
resources for learning, choosing and implementing appropriate learning strategies,
and evaluating learning outcomes. (p. 18)
D’A Slevin and Lavery (1991) argue that this definition (and others) is ambiguous and conceptually difficult to define. Regardless of the myriad definitions that are used to describe SDL, Nolan and Nolan (1997) contend they all describe a process based on the principles of adult education. Another common concept in describing SDL involves learner control in planning and managing the learning (O’Shea, 2003). Merriam et al.
38 (2007) note that the description of the self-directed learner arising from Guglielmino’s
(1978) Delphi study provided the most-used operational definition:
A highly self-directed learner, based on the survey results, is one who exhibits
initiative, independence, and persistence in learning; one who accepts
responsibility for his or her own learning and views problems as challenges, not
obstacles; one who is capable of self-discipline and has a high degree of curiosity;
one who has a strong desire to learn or change and is self-confident; one who is
able to use basic study skills, organize his or her time and set an appropriate pace
for learning, and to develop a plan for completing work; one who enjoys learning
and has a tendency to be goal-oriented. (p. 73)
Further, Brockett and Hiemstra (1991) describe SDL as both an approach of instruction and a preferred trait or skill. As an approach of instruction, they state self- directed learning is “a process in which a learner assumes primary responsibility for planning, implementing, and evaluating the learner process” (Brockett & Hiemstra, p.
24). Regarding self-directed learning as a preferred trait or skill, they describe it as a
“learner’s desire or preference for assuming responsibility for learning” (Brockett &
Hiemstra, p. 24). Candy (1991) suggests that SDL can also be viewed as an outcome of the learning process if self-management and personal autonomy are outcomes.
Knowles (1975) correctly prophesized that:
Education – or, even better, learning – must now be defined as a lifelong process.
The primary learning during youth will be the skills of inquiry and the learning
after schooling is done will be focused on acquiring the knowledge, skills,
39 understanding, attitude, and values required for living adequately in a rapidly
changing world. (p. 16)
Self-Directed Learning in the Medical Field
The medical field was one of the first to embrace SDL as a learning strategy (Box,
1982; Clark, 1991; Dixon, 1989; Guglielmino & Guglielmino, 1984; Holm, 1980;
Hutton, 1985; Katherein, 1981; Lengacher & Van Cott, 1992; Linares, 1987; Linares,
1989; Malin, 1985; Middlemiss, 1989; Moore, 1987; Murray, 1987; O’Kell, 1988;
O’Shea, 2003; Pearson, 1989; Savoie, 1979; Skaggs, 1981; Stipe, 1987; Wiley, 1981).
Nursing, in particular, was one of the first professions to embark on peer-reviewed research in SDL (O’Shea). Using Tough’s interview schedule, Emblen and Gray (1990) found that nurses spent an average of 313 hours per year (217 hours on professional topics and 96 hours on non-professional topics) on SDL learning projects. Interestingly, even though much of nursing curriculum continues to be driven by SDL, the majority of
SDL field research was done in the 1980’s (O’Shea).
Research done with SDL and physician-preparation, however, has continued to expand and delve into new areas. In fact, studies have even demonstrated the effective use of SDL strategies in computerized simulation skills in gross anatomy dissections
(Arroyo-Jimenez et al., 2005). Knowledge acquisition has been measured as statistically and educationally significant in students who attended conferences and used self-directed computer resources that were comparable to the gain in knowledge of a year in residency training (McDonald, Zeger, & Kolars, 2007).
In a study involving three U.S. medical schools (n=941), the SDLRS/LPA was found to be a reliable, gender-fair assessment. The mean score for the medical students
40 was 235, placing them in the above-average range of SDLRS/LPA scores for the general
population (Guglielmino, Mazmanian, Guglielmino, Hoban, & Pololi, 2002).
Previous research conducted with 182 UTMB third-year medical students
revealed a mean SDLRS/LPA score of 235.81, which is higher than the mean score of
227.7 of a 5,000-subject meta-analysis of college students and professionals (McCune,
Guglielmino, & Garcia, 1990), and significantly (p < .05) higher than the mean of the
general population (214) (Shokar, Shokar, Romero, & Bulik, 2002). Self-directed modes
have been identified as the most effective approach to improving physician performance
and patient care outcomes (Candy, 1995; Davis, O’Brien, et al., 1999; Davis, Thompson,
Oxman, & Haynes, 1999; Horn et al., 1997; Mamary & Charles, 2003). A study of
general practice trainees by Bligh (1992) indicated that the factors that affect readiness
for SDL are “enjoyment and enthusiasm for learning, a positive self-concept as a learner and a factor suggesting the possibility of ‘reproducing’ orientation to learning” (p. 497).
While curriculum designed to engage medical students in SDL has expanded, some studies have shown “supportive participation” more conducive to learning and preferred by students (Dornan, Hadfield, Brown, Boshuizen, & Scherpbier, 2005). Miflin
(2004) comments that since much of Knowles’ work identified that SDL appeals to adults
(in their 30’s) who value personal autonomy in their learning, some question whether the typical age of medical students (20-27) qualifies the employment of similar learning methods. In fact, Dornan et al.’s (2005) study suggests that medical students defaulted to
SDL as a method of learning only when the instructor-centered support and guidance were deficient. According to Boud (1998), too often SDL in many university courses
41 “has come to mean independent of classes, independent of other students, or independent
of faculty” (p. 21).
As a final point, it should be noted that physicians are required to maintain levels
of Continuing Medical Education (CME). In fact, the American Board of Medical
Specialists, (ABMS) recently documented a need for monitoring and promoting
continuing competence in practicing medicine (Miller, 2005). As a result, ABMS has
instituted a Maintenance of Certification program, which includes recurring examinations
of knowledge and practice (Miller). This Maintenance of Certification contains four
major components: “professional standing, including an unrestricted license to practice medicine; lifelong learning and self-assessment; demonstrated cognitive expertise; and practice performance assessment” (Miller, p. 151). The proliferation of new information in the information age has made keeping current of best practices for standards of care an even greater component of a physician’s job. With regard to CME, Hiramanek (2005) noted that self-directed learning is the “most effective approach for improving physician performance and patient care outcomes” (p. 879). Candy (1991) states that there are three aspects of the learner’s capacity to be a lifelong professional learner:
The first is the broad self-management competencies that are the basic building
blocks of all independent learning; these include research skills, time
management, goal writing, critical thinking, and so on. The second facet,
frequently overlooked, is sufficient familiarity with the subject matter to be able
to engage in truly self-directed learning. The third and arguably the most elusive
is the attainment of a sense of learning competence – the quiet assurance that one
is able to exercise control effectively in a certain situation. (p.xix)
42 Chapter Summary
A review of the literature regarding physicians and physician preparation,
integrated medical curricula (IMC) including problem-based learning and evidence-based
medicine, the National Board of Medical Examiners Family Medicine Shelf Exam
(NBME-FM), Objective Structured Clinical Examination (OSCE), and self-directed
learning (SDL) were explored. The literature review concludes by underscoring the
importance of self-directed learning as an integral component of physician preparation
and an essential continuing practice for physicians to ensure their growth and change
with the influx of new information and techniques in the field.
Chapter 3 will describe the methodology in this research study. Following a brief introduction, information regarding the subjects, institutional setting, instrumentation, procedures, and data collection and analysis will be highlighted. The chapter will conclude with a summary.
43
CHAPTER 3
METHODOLOGY
This chapter describes the research methods used in this study to address the purpose and investigate the five research questions:
1. Are third-year medical students’ scores on the Self-Directed Learning Readiness
Scale/Learner Preference Assessment (SDLRS/LPA) higher than the scores of the
general adult population as reported by Gugliemino & Gugliemino in 1988 (214.0
+ 25.59)?
2. Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores
and knowledge-based measures of success (National Board of Medical
Examiners’ Family Medicine Shelf Examination [NBME-FM] scores)?
3. Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores
and performance-based measures of success (Objective Structured Clinical
Examination [OSCE] scores, preceptor rating scores)?
4. Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores
and the combination of knowledge-based and performance-based measures of
success (final grade)?
44 5. Are knowledge-based and performance-based measures of success (Objective
Structured Clinical Examination scores, preceptor ratings scores, final grade)
significant in predicting Self-Directed Learning Readiness Scale/Learner
Preference Assessment (SDLRS/LPA) scores and/or National Board of Medical
Examiners Family Medicine Shelf Examination (NBME-FM) scores?
Topics addressed in this chapter include the institutional setting for the study, including
detailed information regarding its curriculum requirements; a description of the subjects;
instrumentation used in evaluation; procedures; data collection; and data analysis.
Institutional Setting for the Study
Galveston, Texas, (population 57,000) is an island located two miles off the
mainland and is home to UTMB. Opened in 1891, UTMB is the oldest medical school in
Texas and one of the largest in the United States (UTMB School of Medicine Bulletin,
2007-2009). Approximately 200 medical doctors graduate yearly from 15 clinical and 5
basic science departments (UTMB School of Medicine Bulletin, 2007-2009). Their
mission is to “provide scholarly teaching, innovative scientific investigation, and state-of-
the-art patient care in a learning environment to better the health of society” (UTMB
School of Medicine Bulletin, p. 4).
The University of Texas Medical Branch served as the state’s only medical school until 1949. In the Houston/Galveston area, UTMB is the third largest employer and
contributes approximately $935 million to the regional economy. The Liaison Committee of Medical Education (LCME) most recently fully accredited UTMB in 1998, shortly after the college transitioned from a traditional medical curriculum to an integrated medical curriculum (IMC) (UTMB Self-Study Summary, 2006).
45 The most recent accreditation by the LCME in 1998 coincided with the initiation
of the IMC for first and second-year students. At this time, the LCME acknowledged ten
areas of strength and several areas in transition. In 2001, these areas of concern were all
successfully addressed and no follow-up reports were required (UTMB Self-Study
Summary, 2006).
In 1998, the adoption of the IMC coincided with the Medical School Objectives
Project of the AAMC. The foundations of the IMC were based on the belief that, “the
primary expectations of the medical profession, that physicians should be knowledgeable,
skilled, possess professional attitudes, and be dedicated to lifelong learning, are the
themes of the educational objectives” (UTMB Self-Study Summary, 2006, p. 10). As a
result, the planning, development, evaluation, and review of the curriculum are based on
these beliefs (UTMB Self-Study Summary).
Applicants to the school must have completed 90 semester hours in a
baccalaureate program with a minimum of a C average (UTMB School of Medicine
Bulletin, 2007-2009). Currently, the mean grade point average of students accepted to
UTMB is 3.73/4.00 (UTMB School of Medicine Bulletin). Students are required to have
taken one year of college English, two years of biology (1 year with formal lab), ½ year
of calculus or ½ year of statistics, one year of physics with lab, one year of general
chemistry and one year of organic chemistry with lab (UTMB School of Medicine
Bulletin, 2007-2009). Additionally, coursework in biochemistry/advanced biochemistry,
cell biology, molecular genetics, anatomy and physiology, immunology, statistics,
developmental biology and microbiology are recommended prior to application (UTMB
School of Medicine Bulletin). Students must also have taken the MCAT and been
46 interviewed by a selection committee prior to acceptance (UTMB School of Medicine
Bulletin).
Students matriculate through the IMC based on goals and objectives set forth by
the Association of American Medical Colleges, (UTMB School of Medicine Bulletin,
2007-2009). These goals are: “to produce knowledgeable physicians, to produce skillful
physicians, to produce physicians possessing professional attitudes, and to produce
physicians committed to lifelong learning” (UTMB School of Medicine Bulletin. p. 12-
13).
The characteristics of the IMC explicitly intended to “aid learning and build
lifelong learning skills” (UTMB School of Medicine Bulletin, 2007-2009. p. 14), are:
1. Basic science material is integrated across disciplines.
2. Basic science material is integrated with clinical science material.
3. The problem-solving challenges are designed to foster independent learning
and build lifelong learning skills. The more actively involved students are in
the learning experience, the more they will retain. This will serve students
well as they continue to learn throughout their careers (UTMB School of
Medicine Bulletin, p. 14).
The first two years of the IMC consist of 24 weeks of a “Practice of Medicine” core scientific principles course fundamental to medical practice, then six organ-based
blocks and a course on “Great Syndromes,” all of which integrate “anatomy, physiology,
pathology, pharmacology, microbiology, and immunology” for the major body systems
(UTMB School of Medicine Bulletin, 2007-2009. p. 14). In these two years, a physician is
assigned to facilitate a group of 6-8 students in a PBL curriculum (Bulik, 2003).
47 According to the UTMB School of Medicine Bulletin, “all courses are interdisciplinary
and are based on self-directed, problem-based learning, with supplemental large-group
lectures and laboratory sessions” (p. 14). Clearly, to undertake the rigor of the first two
years of medical school, a student must possess the readiness for self-direction.
After the second year of studies is completed, students take Step 1 of the USMLE
(UTMB School of Medicine Bulletin, 2007-2009). This multiple-choice examination
covers basic science as it relates to individual organ systems (USMLE, 2007). Typically,
students must score at least 184, which correlates to answering approximately 60-70% of
the questions correctly, in order to pass the exam (USMLE). This examination must be
passed within three attempts or the student will be withdrawn from the curriculum and
typically dismissed (UTMB School of Medicine Bulletin). In 2006, 92% of all students
from U.S. and Canadian schools passed this exam (USMLE).
Since the advent of the IMC, UTMB student performance as measured by
USMLE Steps 1, Step 2-CK, and Step 2-GS have all increased with scores that meet or
exceed the national average. Surveys reveal that students are satisfied with the quality of
education they received. Additionally, questionnaires following residency training reveal
that the majority of students feel that educational objectives have been met and they are
well prepared for residency training (UTMB Self-Study Bulletin, 2006).
The third year of medical school consists of 48 weeks in which clinical instruction
and experiences comprise the overriding learning model (Bulik, 2003; UTMB School of
Medicine Bulletin, 2007-2009). During the fourth year, the USMLE Step 2 Clinical
Knowledge (CK) Examination and USMLE Clinical Skills (CS) Examination are taken.
These multiple-choice exams evaluate the students’ application and comprehension of
48 patient care under supervision and focus on health promotion and preventing disease.
Typically, students must score 184, answering approximately 60-70% of the questions correctly in order to pass the exam (USMLE, 2007). In 2006, 93% and 98% of all students from U.S./Canadian schools passed the Step 2 CK and CS respectively in 2006
(USMLE).
Subjects
Subjects were 873 third-year medical school students from the University of
Texas Medical Branch (UTMB) at Galveston between 2003 and 2007. Data was collected on 992 students in that 5-year period, but 119 students had missing values for either the
SDLRS/LPA and/or NBME-FM so they were eliminated from the study.
While demographic and ethnic information was unavailable, according to the
AAMC (2007), UTMB had 863 students enrolled in 2006; 429 males and 434 females.
The race/ethnicity breakdown of the 183 students who graduated in 2006 is shown in
Table 2.
Table 2
Total Graduates at UTMB by Race/Ethnicity within Gender, 2006.
Native Mexican Other Black Asian White Cuban Foreign Unknown Hawaiian American Hispanic Female 9 23 1 40 7 1 3 1 1 Male 3 12 0 68 9 1 4 2 1 Total 12 35 1 108 16 2 7 3 2
49
Instrumentation
Self Directed Learning Readiness Scale/Learning Preference Assessment (SDLRS/LPA)
The SDLRS/LPA was developed by Guglielmino (1978) and is the most
commonly used valid quantitative tool to measure an individual’s existing readiness for
managing his or her own learning. Based on a three-round Delphi study with a panel of
13 experts, this 58-item, 5-point Likert-scale questionnaire measures the attitudes, skills
and characteristics that encompass the learner’s state of readiness for self-directed
learning. Since the instructions indicate the respondent should not be aware of the name or purpose of the instrument to control for bias, the self-scoring version is referred to as the Learner Preference Assessment (LPA) (Guglielmino & Guglielmino, 1991). Scoring consists of adding the Likert-type responses to 41 of the questions, reverse scoring the other 17 items, and adding the scores. The range of the instrument is between 58 and 290, with the mean score for adults at 214.00 + 25.59. Guglielmino’s (1978) research
concluded that:
A highly self-directed learner, based on the survey results, is one who exhibits
initiative, independence, and persistence in learning; one who accepts
responsibility for his or her own learning and views problems as challenges, not
obstacles; one who is capable of self-discipline and has a high degree of curiosity;
one who has a strong desire to learn or change and is self-confident; one who is
able to use basic study skills, organize his or her time and set an appropriate pace
for learning, and to develop a plan for completing work; one who enjoys learning
and has a tendency to be goal-oriented. (p. 73)
50 A Cronbach alpha internal reliability of .87 is reported by the author of the scale
(Guglielmino, 1978). Comparable internal reliability estimates range from .72 - .92.
Test-retest reliability coefficients have been reported at .82 (Finestone, 1984) and .79
(Wiley, 1981). Guglielmino and Guglielmino (1991) have reported the highest reliability figure (.94, n = 3,151) with a split-half Pearson correlation with a Spearman-Brown
correction factor. The most recent major analysis of reliability (internal consistency, test- retest) and validity (content, construct, criterion-related) was conducted by Delahaye and
Choy (2000), who found the instrument could accurately be utilized with confidence to measure SDL.
The content validity of SDLRS/LPA was confirmed by Finestone (1984), who verified the characteristics found in the initial Delphi study were compatible with a comprehensive review of the existing SDL literature. The construct validity of the
SDLRS/LPA has been confirmed in numerous studies, most notably by Posner (1989), who reported convergent validity (p < .001) with preference for change (.81), curiosity for learning (.79), perceived scholastic competence (.69), use of internal criteria for evaluation (.64), and independent judgment (.54). Divergent construct validity has been found in negative correlation with preference for structure (r =-.35) (Roberts, 1986), dogmatism (r = -.14) (Long & Agyekum, 1984), and power distance (r = -.95)
(Guglielmino & Guglielmino, 2006). In terms of criterion-related validity, Hall-Johnson
(1981) and Hassan (1981) found significant positive correlations with learning projects undertaken, Graeve (1987) found significant relationships with hours spent on SDL activities, and Jones (1990) identified significant positive relationships with observable student behaviors related to SDL readiness.
51 While the SDLRS/LPA has had some critics (Hoban, Lawson, Mazmanian, Best,
& Seibel, 2005); Maltby, Lewis and Hill’s (2000) comprehensive review concluded, “The
LPA can be used with acceptable confidence to provide an accurate measurement of
readiness for self-directed learning.” Merriam et al. (2007) note it is the “most
frequently used quantitative instrument in studies of self-directed learning” (p.121).
Brockett (1985) argued that the SDLRS/LPA may be biased toward learning “related to
schooling and/or learning acquired through books and study skills,” (p.15) and may not be suitable for older adults with little academic experience. Given the age and academic achievement of the medical students in this study, this criticism would not be of concern.
In addition, the SDLRS/LPA has been demonstrated to be a reliable, gender-fair assessment in a previous study of medical students (Guglielmino et al, 2002).
National Board of Medical Examiners Family Medicine Shelf Examination (NBME-FM)
The NBME provides subject tests in the basic and clinical sciences for the purpose of assessing educational achievement (NBME, 2008). The Family Medicine
Shelf Examination (NBME-FM) consists of 100 questions, most of which are single-best answer (a through e) based on lengthy patient-based scenarios. The exam is timed at 2 hours, 10 minutes and focuses on “normal growth and development and general principles of care throughout the lifespan,” (Examination Focus section) and “diagnosis and initial management of common life-threatening and debilitating diseases”
(Examination Focus section) (Southwestern Medical Center, 2008).
The advantages of using the NBME-FM are the universal acceptance of the exam, its ease of administration, and the national norms that allow for comparison (Appel et al.,
2002). The disadvantage of the NBME-FM is it measures only factual knowledge and
52 may not correlate with performance-based examinations (Appel et al.). Additionally,
since it is a multiple choice exam, students may be prompted to guess the correct answer
(Appel et al.). In fact, studies have shown that individual and group scores on multiple
choice exams are between 8% and 12% higher than on free response or essay
examinations (Bloom, 1956). However, it has been reported that guessing on exams that
provide answers does not change rank order in the top 10% or bottom 10% - the two groups most concerned with rank assessment (Appel et al.).
The predictive validity of the NBME-FM for practice performance has also been shown to be high. According to Appel et al. (2002), “one large study looked at the predictive validity of multiple choice examination performance and practice performance
7 to 10 years later (as assessed by peer rating or chart review) and found a true correlation between test performance and clinical performance” (p.4, para 2).
Objective Structured Clinical Examination (OSCE)
A clinical competence examination rating form is used to measure the Objective
Structured Clinical Examination (OSCE). The OSCE is 27-item 5-point Likert scale standard evaluation form covering student performance during an observed patient encounter, laboratory requests, an oral presentation and question/answer period
(Appendix B). Students are assessed on two faculty OSCEs (OSCE 1, OSCE 2) and the
two are averaged together (OSCE AVG). At UTMB, students must pass with a B average.
The OSCE has been shown to be a valid measurement tool with high levels of
face, content and construct validity (Appel et al., 2002). However, reliability coefficients
53 of around 0.80 make it difficult for clerkship directors to make pass/fail decisions based
solely on the OSCE (Appel et al.).
Preceptor Rating
Each student is assigned to one primary preceptor – a faculty member who works
closely with the student and is able to evaluate that student’s ability. If a student spends
sufficient time with multiple preceptors, up to three may be able to evaluate the student.
Each preceptor completes a clerkship evaluation form, a 10-item standard evaluation
form used to rate student performance (Appendix C) for their one-month clinical
experience. Students are assessed in history-taking skills, physical examination skills,
communication skills, problem-solving skills and professionalism. The advantages of this
type of examination are they are a measure of practical competence, rated by experienced
preceptors.
These forms are used to determine a global rating of students but are often viewed
as excessively subjective (Appel et al., 2002). The disadvantages of using this tool are possible lack of training for evaluators, insufficient time for faculty/student interaction and no clear course objectives for the material to be mastered (Appel et al.). One severe limitation of this type of evaluation tool is the inclination to generalize performance based on one trait, as this compromises the validity of the ratings (Appel et al., 2002).
Final Grade
A formula equation is used to take into account variability in faculty observation
(OSCE AVG), evaluation (preceptor AVG) and NBME-FM scores. Scale scores between one and five are assigned (i.e., 3 = 75) and faculty-determined weights are used to determine students’ final grade.
54 Procedures
Data Collection
This was a retrospective study using archived data from a 3-year period.
Consequently, all subject data (n = 873) has been collected at UTMB. The SDLRS/LPA was administered to students under the direction of an Associate Professor in the Office of Educational Development during the third year of their medical preparation.
SDLRS/LPA data collection was supervised by a senior investigator and approved by the
IRB of UTMB. NBME-FM scores, two OSCE scores and the average of the two scores
(OSCE 1, OSCE 2, OSCE AVG), between one and three preceptor ratings and the preceptor ratings average (Preceptor 1, Preceptor 2, Preceptor 3, Preceptor AVG) and final grades were made available to the same investigator. All data was transferred to this investigator in consideration of IRB regulations in an electronic Excel spreadsheet without subject identifiers. Data was transferred to an SPSS spreadsheet for analysis.
Data Analysis
Quantitative data analysis was employed, utilizing SPSS Software (version 17.0,
Chicago, IL) to determine if relationships exist between SDLRS/LPA scores and knowledge-based scores (NMBE-FM), performance-based scores (OSCE and preceptor scores) and the combination of knowledge-based and performance-based scores (final grade). The group mean on the SDLRS/LPA was compared to the published mean score of 214.0 + 25.59 for general adult learners, 235.8 + 19.99 for third-year medical students in a previous study, and other selected previous studies.
Correlation analysis was employed to determine the relative relationship of
SDLRS/LPA scores with measurements of medical school success (NBME-FM scores,
55 OSCE scores, preceptor rating scores, and final grade). These measures of success represent knowledge-based and performance-based measures and a combination of knowledge-based and performance-based measures. Regression analysis was employed to determine if students’ SDLRS/LPA scores were significant predictors of measures of success in medical school (NBME-FM scores, OSCE scores, preceptor rating scores, final grade).
Chapter Summary
This chapter discussed the methodology used in the research study. After a brief introduction, the subjects and the institutional setting for the study were described. The instrumentation used to collect data (SDLRS/LPA, NBME-FM, OSCE, preceptor rating scores, and final grade scores) were discussed. The procedures of the study were described including how the data were collected and analyzed. Chapter 4 will present the findings of the study through statistical analysis. This will be accomplished through mean score comparisons, correlation analyses and regression analyses. Finally, the results of the hypothesis testing will be explored.
56
CHAPTER 4
FINDINGS
The purpose of this study was to investigate (a) the self-directed learning
readiness of third-year medical students in comparison to previously reported scores for
the general population and (b) the relationship between self-directed learning (SDL)
readiness and knowledge-based and performance-based measures of success in a medical
school using an integrated medical curriculum; and (c) to determine if knowledge-based
and performance-based measures of success are significant in predicting SDLRS/LPA and
National Board of Medical Examiners Family Medicine Shelf Examination (NBME-FM) scores. Five research questions guided this inquiry:
1. Are third-year medical students’ scores on the Self-Directed Learning Readiness
Scale/Learner Preference Assessment (SDLRS/LPA) higher than the general adult
population as reported by Guglielmino & Guglielmino in 1988 (214.0 + 25.59)?
2. Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores
and knowledge-based measures of success (National Board of Medical Examiners
Family Shelf Examination scores)?
3. Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores
57 and performance-based measures of success (Objective Structured Clinical
Examination [OSCE] scores, preceptor rating scores)?
4. Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores
and the combination of knowledge-based and performance-based measures of
success (final grade)?
5. Are knowledge-based and performance-based measures of success (Objective
Structured Clinical Examination scores, preceptor ratings scores, final grade)
significant in predicting Self-Directed Learning Readiness Scale/Learner
Preference Assessment (SDLRS/LPA) scores and/or National Board of Medical
Examiners Family Medicine Shelf Examination (NBME-FM) scores?
Data were analyzed using the Statistical Package for the Social Sciences (SPSS)
computer software, Version 17.0. The first section of the findings presents the test data
for the sample, including mean values for SDLRS/LPA scores, NBME-FM scores, OSCE
scores, preceptor rating scores and final grade. The next section presents SDLRS/LPA
scores with comparisons made to the general adult population (Research Question 1), the
results of correlation analyses of SDLRS/LPA scores with the other variables (Research
Questions 2, 3, 4) followed by the regression analyses results for predictive models for
SDLRS/LPA and NBME-FM scores (Research Question 5). The chapter concludes with a summary of the findings.
58 Mean Scores
Self-Directed Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA)
The mean sample score on the SDLRS/LPA was 229.06 + 23.19 (n = 873). The possible range of the instrument is from 58 to 290. The range was 173 with a minimum of
113 and a maximum of 286. The median was 229 and the mode was 240. The skewness was -.13 and the kurtosis was .35, indicating a normal distribution. Figure 2 shows the frequency distribution of scores. Twenty-two students fell at or below the 10th percentile
(183), while 184 students scored above the 90th percentile (249).
100
80
y 60 c n e u q e r F
40
20
Mean = 229.0641 Std. Dev. = 23.18826 N = 873 0 100.00 150.00 200.00 250.00 300.00 SDLRS/LPA
Figure 2. Mean Self Directed Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) frequency.
59 National Board of Medical Examiners Family Medicine Shelf Exam (NBME-FM)
The mean sample score on the NBME-FM was 71.58 + 7.60. The range was 48.00 with a minimum of 48.00 and a maximum of 96.00. The median was 71.00 and the mode was 69.00. The skewness was .16 and the kurtosis was .02, indicating a normal distribution. Figure 3 shows the frequency distribution of scores.
100
80
y 60 c n e u q e r F
40
20
Mean = 71.5808 Std. Dev. = 7.59606 N = 873 0 40.00 50.00 60.00 70.00 80.00 90.00 100.00 NBME
Figure 3. Mean National Board of Medical Examiners Family Medicine Shelf Exam
(NBME-FM) scores.
60 Objective Structured Clinical Examination (OSCE)
Students are assessed on two faculty-based Objective Structured Clinical
Examinations (OSCE 1 and OSCE 2). Those scores, along with the average of the two scores (OSCE AVG), are represented here.
The mean sample score on the OSCE 1 was 87.69 + 5.50. The range was 36.22 with a minimum of 60.68 and a maximum of 96.90. The median was 88.70 and the mode was 84.00. The skewness was -1.28 and the kurtosis was 2.64.
The mean sample score on the OSCE 2 was 88.41 + 5.30. The range was 38.83 with a minimum of 57.52 and a maximum of 96.35. The median was 89.39 and the mode was 84.00. The skewness was -1.38 and the kurtosis was 3.16.
The mean sample score for the OSCE AVG was 88.05 + 4.31. The range was
25.86 with a minimum of 70.14 and a maximum of 95.99. The median was 86.33 and the mode was not reported. The skewness was -1.02 and the kurtosis was 1.29, indicating that most students perform extremely well on the OSCE. Figure 4 shows the distribution of scores.
61 80
60 y c n e u
q 40 e r F
20
Mean = 88.0475 Std. Dev. = 4.3131 N = 873 0 70.00 75.00 80.00 85.00 90.00 95.00 100.00 OSCE AVG
Figure 4. Mean Objective Structured Clinical Examination (OSCE) scores.
Preceptor Ratings
Students are assigned to between one and three preceptors who complete assessments for their one-month clinical experience. The mean score on the preceptor 1 rating was 93.23 + 3.06. The range was 32.06 with a minimum of 65.54 and a maximum of 97.60. The median was 93.7. The skewness was -1.86 and the kurtosis was 8.56. All
873 students were evaluated by an initial preceptor.
The mean score on the preceptor 2 rating was 92.47 + 3.83. The range was 25.96 with a minimum of 70.74 and a maximum of 96.70. The median was 92.85. The
62 skewness was -2.20 and the kurtosis was 7.85. One hundred sixty-one students were evaluated by a second preceptor.
The mean score on the preceptor 3 rating was 92.31 + 2.88. The range was 11.25 with a minimum of 85.45 and a maximum of 96.70. The median was 92.5. The skewness was -.34 and the kurtosis was -.65. Forty-two students were evaluated by a third preceptor.
The mean score on the average preceptor rating was 93.22 + 2.98. The range was
29.46 with a minimum of 68.14 and a maximum of 97.60. The median was 93.6. The skewness was -1.74 and the kurtosis was 6.97, indicating that most students perform extremely well in their clinical experience. Figure 5 shows the frequency distribution of scores.
63 150
100 y c n e u q e r F
50
Mean = 93.219 Std. Dev. = 2.98198 N = 873 0 70.00 80.00 90.00 100.00 Preceptor Rating Average
Figure 5. Mean preceptor rating scores.
Final Grade
A final reported student grade is reported based on a formula equation to take into account variability in faculty observation and evaluation in the OSCE, preceptor rating and NBME-FM. The mean score on the final grade was 85.16 + 5.36. The range was
42.19 with a minimum of 54.25 and a maximum of 96.44. The median was 85.98. The skewness was -1.03 and the kurtosis was 1.87 indicating most students get extremely good final grades. Figure 6 shows the frequency distribution of scores.
64 120
100
80 y c n e u
q 60 e r F
40
20
Mean = 85.1648 Std. Dev. = 5.35528 N = 873 0 50.00 60.00 70.00 80.00 90.00 100.00 Final Grade
Figure 6. Mean final grade scores.
Research Questions
Research Question 1
Are third-year medical students’ scores on the Self-Directed Learning Readiness
Scale/Learner Preference Assessment (SDLRS/LPA) higher than the general adult population (214.0 + 25.59)? The mean SDLRS/LPA score was 229.06 + 23.19, which places the sample in the top 31 % of those tested (Guglielmino & Guglielmino, 1991).
The mean is slightly above a meta-analysis of professionals and students (McCune et al.,
65 1990) and slightly below a previous study of third-year medical students (Shokar et al.,
2002). Table 3 shows the mean compared to selected previous studies.
Table 3
Mean SDLRS Scores for Third-Year Medical Students and Previously Reported Samples. ______
Group tested Researcher(s), year Mean + SD
Exemplary elementary principals (Guglielmino & Hillard, 2007) 267.8 + 11.5
Top US female executives (Guglielmino, 1996) 257.8 + 14.7
Top US entrepreneurs (Guglielmino & Klatt, 1994) 248.6 + 18.7
Medical supervisors and managers (Muller, 2007) 243.6 + 20.4
Parks and recreation certified professionals (Bryan-Wunner, 1991) 240.67 + 21.4
YMCA directors (Zsiga, 2007) 236.1 + 20.7
Third-year medical students (Shokar et al., 2002 235.8 + 20.0
Current study third-year medical students (Findley, 2009) 229.1+ 23.2
Meta-analysis of professionals and students (McCune et al., 1990) 227.7*
General adult population (Guglielmino & Guglielmino, 1988) 214.0 + 25.6
______
Self-Directed Learning Readiness Scale/Learner Preference Assessment
(SDLRS/LPA) scores have been categorized between Low and High. Table 4 shows the breakdown of the number of third-year medical students in each category.
66 Table 4
Number of Third-Year Medical Students by SDLRS/LPA Category
Readiness for SDL Score # of students
Low 58-176 12 Below Average 177-201 85 Average 202-226 302 Above Average 227-251 318 High 252-290 156
The first hypothesis was tested by comparing the SDLRS/LPA mean in the current study (229.06 + 23.19) to the previously reported obtained mean for general adult learners as reported by Guglielmino & Guglielmino in 1988 (214.0 + 25.59). Independent t-test results show that the third year medical students were significantly (p < 0.05) higher than the general adult population. Null Hypothesis 1 stated: Scores of third-year medical students’ Self-Directed Learning Readiness Scale/Learner Preference Assessment
(SDLRS/LPA) scores are not higher than the general adult population (214.0 + 25.59).
Null hypothesis 1 was rejected.
Research Question 2
Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores and knowledge-based measures of success (National Board of Medical Examiners Family
Shelf Examination scores)? The second hypothesis was tested by computing Pearson r correlations, which showed significant (p < 0.05) relationships between SDLRS/LPA scores and NBME-FM scores (r = .073).
67 Null Hypothesis 2 stated: There is no significant correlation between third-year medical students’ Self-Directed Learning Readiness Scale/Learner Preference
Assessment (SDLRS/LPA) scores and knowledge-based measures of success (National
Board of Medical Examiners Family Medicine Shelf Examination [NBME-FM] scores).
Null hypothesis 2 was rejected.
Research Question 3
Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores and performance-based measures of success (Objective Structured Clinical Examination scores, preceptor rating scores)?
The third hypothesis was tested by computing Pearson r correlations, which showed significant (p < 0.01) relationships between SDLRS/LPA and OSCE 1 (r = .109, p
= .001), OSCE 2 (r = .103, p = .002), and OSCE AVG (r = .133, p = .000). Correlation analysis also showed significant (p < 0.05) correlations between SDLRS/LPA scores and preceptor rating 2 (r = .168, p = .034), and preceptor rating 3 (r = .305, p = 0.05). While these correlations show statistical significance, they should be interpreted with caution since the effect size, in most cases, indicate a trivial or small effect. No significant correlation existed between SDLRS/LPA scores and preceptor rating 1 (r = .041, p = .223) or preceptor rating average (r = .066, p = 0.50).
Null Hypothesis 3 stated: There is no significant correlation between third-year medical students’ Self-Directed Learning Readiness Scale/Learner Preference
Assessment (SDLRS/LPA) scores and performance-based measures of success (Objective
68 Structured Clinical Examination scores, preceptor rating scores). Null hypothesis 3 was
rejected.
Research Question 4
Is there a significant correlation between medical students’ Self-Directed
Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA) scores and the
combination of knowledge-based and performance-based measures of success (final
grade)? Correlation analysis showed significant (p < 0.01) correlations between
SDLRS/LPA scores and final grade (r = .138). While this correlation is significant, it
should be interpreted with caution since the effect size is small.
Null Hypothesis 4 stated: There is no significant correlation between third-year
medical students’ Self-Directed Learning Readiness Scale/Learner Preference
Assessment (SDLRS/LPA) scores and the combination of knowledge-based and performance-based measures of success (final grade). Null hypothesis 4 was rejected.
Research Question 5
Are knowledge-based and performance-based measures of success (Objective
Structured Clinical Examination scores, preceptor ratings scores, final grade) significant
in predicting Self-Directed Learning Readiness Scale/Learner Preference Assessment
(SDLRS/LPA) scores and/or National Board of Medical Examiners Family Medicine
Shelf Examination (NBME-FM) scores?
Regression analyses were performed to determine the nature of the relationship
between the variables. The regression equation used SDLRS/LPA scores as the dependent
variable and NBME-FM scores, OSC AVG, and preceptor rating average as variables for
predictors. The final grade was purposely not included in this analysis since NBME-FM
69 scores were calculated into that score and it would confound the results due to colinearity. Results are listed below in Table 5. The r value for the model was .144 and the F ratio was 6.136.
Table 5
Predictor Variables for Self-Directed Learning Readiness Scale/Learner Preference Assessment (SDLRS/LPA)
Beta t Predictor Variables for Self-Directed Learning Readiness and:
NBME-FM 0.033 .946 OSCE AVG 0.115 3.244* Preceptor Rating 0.045 1.326 *Significant at the .001 level
Simple correlations between the dependent variable (SDLRS/LPA) and all predictor variables (NBME-FM, OSCE AVG, preceptor rating) were found to be significant as noted in the previous results. All predictor variables were found to be significant with each other at the p < .01 level. The total model predicted 2.1% of the variation in SDLRS/LPA, which was significant at the p < .01 level. Only OSCE AVG contributed significantly (p < .001) to the model having all other variables present.
The regression equation used NBME-FM scores as the dependent variable and
SDLRS/LPA scores, OSC AVG, and preceptor ratings as variables for predictors. The final grade was purposely not included in this analysis since NBME-FM scores were calculated into that score and it would confound the results due to colinearity. Results are listed below in Table 6. The r value for the model was .312 and the F ratio was 31.273.
70 Table 6 Predictor Variables for National Board of Medical Examiners Family Medicine Shelf Exam (NBME-FM)
Predictor Variables for NBME-FM and: Beta t SDLRS/LPA 0.071 2.104 OSCE AVG 0.295 8.964* Preceptor Ratings 0.050 1.529 *Significant at the .001 level
Significant (p < .01) correlations were found between the dependent variable
(NBME-FM) and two of the independent variables (OSCE AVG, preceptor rating). The correlation between NBME-FM and SDLRS/LPA was significant at the p < .05 level. All predictor variables were found to be significant with each other. Correlations between
OSCE AVG and preceptor rating; and SDLRS/LPA and OSCE were significant at p < .01.
The correlation between SDLRS/LPA and preceptor rating was significant at p < .05. The total model predicted 9.7% of the variation in NBME-FM, which was significant at the p
< .01 level. Only OSCE AVG contributed significantly (p < .001) to the model having all other variables present.
Null Hypothesis 5 stated: Knowledge-based and performance-based measures of success (Objective Structured Clinical Examination scores, preceptor rating scores) are not significant in predicting Self-Directed Learning Readiness Scale/Learner Preference
Assessment (SDLRS/LPA) scores and National Board of Medical Examiners Family
Medicine Shelf Examination (NBME-FM) scores. The null hypothesis was rejected.
71 Chapter Summary
Analysis of 873 third-year medical students resulted in mean scores of 229.06 +
23.19 for the SDLRS/LPA, 71.58 + 7.60 for the Family Medicine Shelf Exam (NBME-
FM), 88.05 + 4.31 for the Objective Structured Clinical Examination AVG, 93.22 + 2.98
for the preceptor rating average, and 85.16 + 5.36 for the final grade.
Correlations were significant for SDLRS/LPA scores to NBME-FM scores, OSCE
scores, preceptor rating scores and final grade. While these correlations were significant, all correlations showed a trivial to small effect size with the exception of SDLRS/LPA and preceptor rating 3, which showed a moderate effect. Regression analysis revealed that
NBME-FM, OSC AVG and preceptor ratings predicted 2.1% of the variance in
SDLRS/LPA, which was significant at the p < .001 level. Regression analysis revealed that SDLRS/LPA, OSC AVG and preceptor ratings predicted 9.7% of the variance in
NBME-FM, which was significant at the p < .001 level.
This chapter presented the findings of the study through statistical analysis. This was accomplished through mean score comparisons, correlation analyses and regression
analyses. The results of the hypothesis testing were discovered and a summary of the findings were given. Chapter 5 will summarize the research study, present and discuss the conclusions resulting from the statistical analyses, delineate the implications and significance of the study, and propose recommendations for future study.
72
CHAPTER 5
CONCLUSIONS, DISCUSSION AND RECOMMENDATIONS
The purpose of this study was to investigate (a) the self-directed learning
readiness of third-year medical students in comparison to previously reported scores for
the general population and (b) the relationship between self-directed learning (SDL)
readiness and knowledge-based and performance-based measures of success in a medical
school using an integrated medical curriculum; and (c) to determine if knowledge-based and performance-based measures of success are significant in predicting SDLRS/LPA and
National Board of Medical Examiners Family Medicine Shelf Examination (NBME-FM) scores. The fact that students were engaged in an integrated medical curriculum (IMC), where problem-based and evidence-based learning strategies that promote self-directed learning were the primary method of instruction was of particular interest.
The Self-Directed Learning Readiness Scale/Learner Preference Assessment
(SDLRS/LPA) was used to measure readiness for self-directed learning and the National
Board Medical Examiners Family Medicine Shelf Exam (NBME-FM) was used to measure medical knowledge in students’ third year. Practical clinical competence was measured by Objective Structured Clinical Examinations (OSCE) and preceptor ratings.
Students’ final grade was a composite of these knowledge-based and performance-based measures.
73 Discussions regarding the relationships of SDLRS/LPA scores with each of the
measures of success in medical school and predictive models for SDLRS/LPA and
NBME-FM will begin the chapter. This will be followed by a summary of the research
study, conclusions resulting from the statistical analysis, and the implications and
significance of the study. Finally, recommendations for future study will be proposed.
Conclusions and Discussion
The medical field was one of the first to embrace self-directed learning as an
educational strategy (Box, 1982; Clark, 1991; Dixon, 1989; Guglielmino & Guglielmino,
1984; Holm, 1980; Hutton, 1985; Katherein, 1981; Lengacher & Van Cott, 1992;
Linares, 1987; Linares, 1989; Malin, 1985; Middlemiss, 1989; Moore, 1987; Murray,
1987; O’Kell, 1988; O’Shea, 2003; Pearson, 1989; Savoie, 1979; Skaggs, 1981; Stipe,
1987; Wiley, 1981). Problem-based learning and evidence-based medicine were adopted
by many medical schools in addition to or in lieu of traditional lecture-based instruction
to accommodate the belief that self-directed learning promotes continued competency in
physicians. Accordingly, many of medicine’s accreditation and licensing organizations
have recognized SDL as a critical attribute in both physician preparation and continuing
education.
Lujan and DiCarlo (2006) point out studies that indicate the quantity of
undergraduate science education has no effect on medical students’ academic success, an outcome they suggest is related to traditional teaching approaches. While the interpretations of these results may be problematic, those responsible for curriculum development are now seeking to find a balance between factual content knowledge and
teaching the ability to be independent, self-directed learners. Since curriculum decisions
74 based on PBL/EBM have revolutionized medical education, it is incumbent upon
researchers and educators to examine SDL in the context of physicians’ professional
preparation.
While research has been conducted on the self-directed learning readiness of
physicians, little research has addressed the relationship SDL readiness has with both
knowledge-based and performance-based measures of medical school success. As a
result, this study sought to explore those relationships by examining third-year medical
students’ levels of readiness for SDL and other measures typical of third-year students.
Levels of Self-Directed Learning Readiness of Medical Students
The results of this study support previous findings that medical students’ mean
levels of self-directed learning readiness are higher than the general population mean
(Guglielmino et al., 2002). It would appear that this field attracts internally-motivated
self-directed learners rather than externally-motivated teacher-centered learners as
Bradley et al. (2005) describe. High levels of SDL are requisite to those who practice
medicine in order to grow as professionals and provide the highest quality of patient care
(Ainoda et al., 2005; Williams, 2004). Research has indicated that those preparing to
become physicians are highly self-directed (Shokar et al., 2002; Pilling-Cormick & Bulik,
1999). Ninety percent of students in this study scored in the average, above average, or
high categories on the Self-Directed Learning Readiness Scale/Learner Preference
Assessment (SDLRS/LPA).
In this study, third-year medical students’ (n = 873) readiness for SDL as
measured by the SDLRS/LPA was 229.19 + 23.49, falling into the above average range.
The mean was significantly (p < .05) higher than the general adult population mean of
75 214.0 + 25.59 (Guglielmino & Guglielmino, 1988). Several previous studies have
measured populations that scored higher than the medical students in this study: for
example, exemplary school principals recognized at the state level (Guglielmino &
Hillard, 2007), top US female executives (Guglielmino, 1996), top U. S. entrepreneurs
(Guglielmino & Klatt, 1994), medical supervisors and managers (Muller, 2007), and
YMCA directors (Zsiga, 2007). All of these samples were much smaller, however, and
the first three consisted of elite groups of recognized high achievers. The mean in this study was lower than the mean of 234.68 reported for 941 students from three U.S. medical schools (Guglielmino et al., 2002), where curricular methods were not identified.
The mean in this study was slightly higher than the mean of a meta-analysis of a sample of more than 3,000 professionals and graduate students with a variety of majors (McCune et al., 1990).
In a previous smaller study with 182 third-year medical students at UTMB
(Shokar et al., 2002), the SDLRS/LPA mean score of 235.81 + 19.99 was also
significantly (p < .01) higher than the general adult population. Therefore, these findings
remain consistent for third-year students at this medical school within this curriculum.
Students in this study were engaged in an IMC, which might be expected to
increase levels of SDL. Whether students’ level of SDL increased in this study due to the
fact they were engaged in an IMC, while probable, is not measurable; SDLRS data from
the prior years was not available. In addition, it is important to note that many variables
besides the IMC curriculum could explain the high SDLRS/LPA scores. Certainly, one of
the limitations of interpreting this data is that medical students’ levels of SDL are already above average and that they would be likely to thrive in any curriculum format. Any
76 attempt to measure improvement may be hindered by a ceiling effect, as was noted by
Caffarella & Caffarella (1986) in a study of graduate students.
SDLRS/LPA Scores and NBME-FM
Significant relationships existed between SDLRS/LPA scores and National Board of Medical Examiners Family Medicine Shelf Examination scores (NBME-FM) (r = .071, p < .05). However, this was a weak association. The NBME-FM, based on extensive patient-based scenarios, is the capstone standardized examination taken by third-year students. While standardized tests such as the NBME-FM measure a “snapshot of content knowledge,” the SDLRS/LPA measures readiness for self-directed learning, which is more aligned with the process by which students learn (Shokar et al., 2002).
Previous studies have suggested the disadvantage of the NBME-FM is it measures only factual knowledge and may not correlate with performance-based examinations
(Appel et al., 2002). In this study, simple correlations between NBME-FM and performance-based examinations (OSCE AVG) were significant at p < 0.01.
SDLRS/LPA Scores and OSCE
The OSCE has been shown to be highly reliable in the assessment of clinical competence (Cohen et al., 1990; Matsell et al., 1991; Petrusa et al., 1990; Sloan et al.,
1995) and measures the qualities in a physician that matter most to patients: humanistic behavior and real-world performance (Dornan et al., 2004). Malloy et al. (1998) warns that OSCE scores may inflated, which may skew correlations to be higher than they actually are.
Significant relationships existed between SDLRS/LPA and Objective Structured
Clinical Examination 1 (OSCE 1) scores (r = .102, p < .0 1), and Objective Structured
77 Clinical Examination 2 (OSCE 2) (r = .101, p < .01). Similarly, Bulik (2003) found
significant (p < 0.05) correlations between SDLRS/LPA and OSCE in a Medical
Ambulatory Clerkship (r = 0.136).
Previous studies have shown OSCE correlates very highly with ABSITE scores.
OSCE has also shown positive correlations with two other standardized medical school
examinations. Simon et al. (2002) found a significant (p < 0.001) correlation between
OSCE and USMLE Step 1 scores (r = 0.41). In 2007, Simon et al., found a significant (p
< 0.001) correlation between OSCE and USMLE Step 2 scores (r = 0.385).
SDLRS/LPA Scores and Preceptor Ratings
Each student is assigned to one primary preceptor – a faculty member who works closely with the student, and is able to evaluate that student’s ability. If a student spends sufficient time with multiple preceptors, up to three may be able to evaluate the student.
Significant relationships existed between SDLRS/LPA and preceptor rating 2 (r = .168, p
= .034), and preceptor rating 3 (r = .205, p < 0.05). However, no significant relationship existed between SDLRS/LPA scores and preceptor rating 1 or preceptor rating average.
This finding is puzzling, as a previous study with 182 third-year medical students at the same university (Shokar et al., 2002), significant (p < .01) correlations were reported between SDLRS/LPA and preceptor ratings in the family medicine clerkship (r = .251) and multidisciplinary ambulatory clerkship (r = .242) (Shokar et al.). No correlations were found between SDLRS/LPA and OSCE in that study (Shokar et al.). Bulik’s study of
560 third-year students also demonstrated significant (p < 0.01) correlations with
SDLRS/LPA and preceptor rating scores in both a Family Medicine clerkship (r = 0.118) and Medical Ambulatory clerkship (r = 0.190).
78 SDLRS/LPA Scores and Final Grade
The correlation between SDLRS/LPA and final grade (r = .133, p = .001) was
also significant, but a weak association existed. The null hypothesis was rejected. In a
previous study with 182 third-year medical students at the same university (Shokar et al.,
2002), significant (p < .05) correlations existed between SDLRS/LPA and final grade in a
multidisciplinary ambulatory clerkship final grade (r = .173). Bulik (2003) also found
significant correlations between SDLRS/LPA and final grade in a Family Medicine
Clerkship (r = 0.092, p < 0.05) and Medical Ambulatory Clerkship (r = 0.187, p < 0.01).
Further Discussion
While the SDLRS/LPA scores of medical students in this study with knowledge-
based and performance-based examinations were modest, they mirror the relationships
that have appeared consistently across a number of studies and indicate a tendency for medical students with higher levels of self-directed learning to perform better in medical preparation programs. Although correlation coefficients in this study were small, they were all positive and many measures reached statistical significance.
The MCAT, perhaps the most-commonly used assessment for medical school entrance, has been shown in a recent meta-analysis to have only small to medium predictive validity for both medical school performance and USMLE (Donnon et al.,
2007). While the MCAT is not the lone factor in selecting applicants, experts have called for additional screening and selection tools to enhance the probability of success. Medical schools also realize that knowledge-based exam scores are only part of the process.
Consequently, further investigation of test instruments that measure qualities and traits defined as being essential to practice should be examined to assess their predictive value.
79 While undergraduate grade point average and MCAT scores are good indicators of
NBME-FM performance, they are still not valuable in predicting clinical performance
(Silver & Hodgson, 1997). Additionally, curriculum changes explicitly designed to enhance SDL should be investigated to determine if the desired effects are being realized.
The SDLRS/LPA contributes to the holistic assessment of those preparing for the medical profession. The assessment of medical knowledge and the ability to apply it are, of course, critical; however, the newer standards for medical education have also addressed the importance of promoting self-directed learning readiness in physician preparation programs. The SDLRS/LPA adds an important dimension to the assessment of medical students, addressing the emphasis on ensuring that physician preparation programs produce practitioners who are likely to be continuing, lifelong learners.
Limitations
This study sought to explore the relationship between self-directed learning readiness and knowledge-based and performance-based measures of success in third-year medical students. Since this study took place at a one large medical school, findings and conclusions must be considered localized to that university and not generalizeable to all third-year medical students.
There is a potential attenuation of the correlation between NBME-FM and performance measures due to the restricted variance of the NBME-FM. Additionally, the
SDLRS/LPA scores may be truncated as well. It must also be assumed that all students answered the SDLRS/LPA truthfully and that variables not investigated were equal across the sample.
80 Another limitation is that there may be undue bias in the student population towards preferring to learn in a self-directed manner. White (2007) has shown that some medical students choose PBL curricula based on their individual learning preference.
Therefore, this preference may differentiate characteristics in these students from other medical school populations.
Recommendations for Future Study
Based on the findings of this study and the associated literature review, several directions for future research on the relationships between self-directed learning to standardized and practical measures of success in third-year medical students are suggested.
While this data adds to the growing body of evidence that SDLRS/LPA scores are a valuable tool in assessing knowledge-based and performance-based outcomes in third- year medical students, further research should focus on other forms of assessment indicative of success. Investigation into the relationship between SDLRS/LPA scores and other knowledge-based medical examinations (USMLE Step 1 and Step 2 scores and
MCAT scores) would further contribute to the understanding of these relationships.
Investigations into the relationship between SDRS/LPA and other performance-based indicators (multiple clerkship grades, grade point average etc.) would be interesting as well. Slaughter (2009), for example, in a longitudinal study of pharmacy students, found
SDLRS/LPA scores were positively related to on-time graduation and grade-point average.
81 Longitudinal studies throughout a student’s medical school experience to
determine whether SDL is fostered by the presence of an IMC would be of interest to
practitioners and administrators, particularly those responsible for designing and
evaluating course curriculum. Blumberg (2000) reported evidence that engaging students
in PBL curricula promoted SDL skills. While students in Williams’ (2004) study
demonstrated no significant changes in SDLRS/LPA scores from baseline to one year of
medical school, the time element may have been insufficient to note significant changes.
Due to the short duration of baseline to final measurement, this would seem to support the argument of Dolmans and Schmidt (1994) that SDL may be a function of both time and experience. Moreover, the level of SDL readiness once students are involved in their
residency program and then into their clinical practice would help complete the picture of
the role of SDL throughout a medical career. Since continuing education plays such a
pivotal role in medical education, future studies should target the role of SDL in the attainment of CMEs.
Those who rate students in the assessment process are worthy of study. Since the
OSCE has been shown to be a valuable tool in the assessment of medical students, further research into the training process of those who rate students may prove to be an important component of this evaluation. Additionally, research examining the SDL readiness of preceptors and their training may be an integral aspect of the process of assessing students.
Since undergraduate education may be a key component in this process, an examination of whether prior preparation moderates the relationship between
82 SDLRS/LPA and the knowledge-based and performance-based assessments would be
important.
Information that further identifies the student population (gender ethnicity, age,
country of origin etc.) would be interesting to evaluate. In Guglielmino et al.’s (2002),
study of more than 900 medical students, no differences in SDLRS/LPA scores existed
between gender, but significant (r = .14, p < .01) correlations existed by age.
Further studies looking at self-directed learning in students involved in an
integrated medical curriculum compared to students in a traditional lecture-based
curriculum would add to the body of knowledge in this area. However, Albanese (2000)
cautions that unreasonable expectations in effect sizes may yield little difference on
knowledge or clinical skills. Another confounding factor is that this comparison would be difficult to interpret because learners who are highly self-directed can self-direct themselves even in formal instructional settings (Brockett et al., 2008). In other words, students who are highly self-directed will seek opportunities for self-direction regardless of method of instruction.
The model of knowledge-based and performance-based assessment used in medical schools may be worthy of study in other professional preparation programs. For example, the possible implications for educational leadership programs using this model would be interesting to explore.
Originally, this study included a follow-up focus group component designed to add richness and meaning to the quantitative data. Unfortunately, Hurricane Ike made direct landfall on Galveston Island on September 13, 2008, and the focus groups for the remainder of the school year were cancelled. The researcher was to ask a set of highly
83 structured, predetermined, open-ended interview questions to these previously- determined focus groups at the end of their third-year clerkships and allow for an opportunity to engage in group dialogue. A future study that enriched the quantitative data with qualitative data could provide new insights.
Summary
One of the major tenets of medical education is to cultivate competencies, such as
SDL, that will transfer into lifelong independent learning (Whittle & Murdoch-Eaton,
2001). In 2001, the AAMC conducted a cross-validation study to identify relevant academic and non-academic attributes of medical students and residents (Etienne &
Julian, 2001). The researchers found that these behaviors that were demonstrated through critical incident reports were: shaping the learning process (“taking an active role in their own learning and knowledge acquisition,” para. 6), self-management and coping skills
(“balancing the demands of medical school with other aspects of life by prioritizing, setting time limits, adapting to diverse environments, and appropriately requesting feedback and assistance from professors or other students,” para. 6), fostering a team environment, interpersonal skills and professionalism, interacting with patients and families, technical knowledge and skill, ethical behavior, mentoring and educating students, and maintaining calm under pressure (Etienne & Julian).
Accordingly, Harvey et al. (2003) state that, “becoming an independent and self- directed lifelong learner is one of the critical outcomes of undergraduate medical education” (p. 1259). Bulik (2003) agrees, stating “Success in medical school is strongly related to the ability to direct and regulate one’s own learning experience” (p. 76). As a result, many professional groups (American Board of Medical Specialists, Accreditation
84 Council for Graduate Medical Education, Association of American Medical Colleges)
have directly or indirectly recognized the importance of promoting self-directed learning.
This investigation of self-directed learning in medical education was unique in
two ways. First, the number of subjects who participated is among the highest of any
study done with SDL and medical students to date. Second, previous studies with medical
students have looked at the relationships between SDLRS/LPA scores and one or two of
three variables: knowledge-based, performance-based or a combination of knowledge-
based and performance-based measures. This may be the first study to look at
SDLRS/LPA scores and all three variables in third-year medical students.
The positive relationships of the SDLRS/LPA scores of medical students in this
study with knowledge-based and performance-based practical examinations were modest, but they indicate a tendency for medical students with higher levels of readiness for self- directed learning to perform better in medical preparation programs. These results reflect relationships that have appeared consistently across a number of other studies. The
SDLRS/LPA adds an important dimension to the assessment of medical students, addressing the emphasis on ensuring that physician preparation programs produce practitioners who not only possess current medical knowledge and the ability to apply it, but are also likely to continue to seek out new knowledge and techniques as the practice of medicine rapidly changes over their careers.
The proliferation of new information in the information age has made keeping current of best practices for standards of care an even greater component of a physician’s job. In fact, it has been estimated that the half-life of biomedical knowledge is 7-10 years
(Rugh, Goggins, & Hatch, 2009). According to Friedman et al. (2005), “the exponential
85 growth of biomedical knowledge and shortening half-life of any single item of knowledge both suggest that modern medicine will increasingly depend on external knowledge to support practice and reduce errors” (p. 334). Physicians will be increasingly reliant on self-directed learning in order to maintain proper levels of care.
86
APPENDIX A
Final Grade Formula
87 let I1 = ITEM1 let I2 = ITEM2 let I3 = ITEM3 letI4 = ITEM4 let I5 = ITEM5 let I6 = ITEM6 let I7 = ITEM7 let I8 = ITEM8 letI9 = ITEM9 If I1 = 5.000 then let I1 = 97 If I1 = 4.000 then let I1 = 85 If I1 = 3.000 then let I1 = 75 If I1 = 2.000 then let I1 = 25 If I1 = 1.000 then let I1 = 85 If I1 = . then let I1 = 85 If I2 = 5.000 then let I2 = 97 If I2 = 4.000 then let I2 = 85 If I2 = 3.000 then let I2 = 75 If I2 = 2.000 then let I2 = 25 If I2 = 1.000 then let I2 = 85 If I2 = . then let I2 = 85 If I3 = 5.000 then let I3 = 97 If I3 = 4.000 then let I3 = 85 If I3 = 3.000 then let I3 = 75 If I3 = 2.000 then let I3 = 25 If I3 = 1.000 then let I3 = 85 If I3 = . then let I3 = 85 If I4 = 5.000 then let I4 = 97 If I4 = 4.000 then let I4 = 85 If I4 = 3.000 then let I4 = 75 If I4 = 2.000 then let I4 = 25 If I4 = 1.000 then let I4 = 85 If I4 = . then let I4 = 85 If I5 = 5.000 then let I5 = 97 If I5 = 4.000 then let I5 = 85 If I5 = 3.000 then let I5 = 75 If I5 = 2.000 then let I5 = 25 If I5 = 1.000 then let I5 = 85 If I5 = . then let I5 = 85 If I6 = 5.000 then let I6 = 97 If I6 = 4.000 then let I6 = 85 If I6 = 3.000 then let I6 = 75 If I6 = 2.000 then let I6 = 25 If I6 = 1.000 then let I6 = 85 If I6 = . then let I6 = 85
88 If I7 = 5.000 then let I7 = 97 If I7 = 4.000 then let I7 = 85 If I7 = 3.000 then let I7 = 75 If I7 = 2.000 then let I7 = 25 If I7 = 1.000 then let I7 = 85 If I7 = . then let I7 = 85 If I8 = 5.000 then let I8 = 97 If I8 = 4.000 then let I8 = 85 If I8 = 3.000 then let I8 = 75 If I8 = 2.000 then let I8 = 25 If I8 = 1.000 then let I8 = 85 If I8 = . then let I8 = 85 If I9 = 5.000 then let I9 = 97 If I9 = 4.000 then let I9 = 85 If I9 = 3.000 then let I9 = 75 If I9 = 2.000 then let I9 = 25 If I9 = 1.000 then let I9 = 85 If I9 = . then let I9 = 85 Let SCORE = (I1*.15)+(I2*.15)+(I3*.10)+(I4*.15)+(I5*.15)+(I6*.15)+(I7*.05)+(I8*.05)+(I9*.05)
89
APPENDIX B
Objective Structured Clinical Examination (OSCE) Form
90 AY 06-07 1 $•∞°≤¥≠•Æ¥ ض &°≠©¨π -•§©£©Æ• 4®• 5Æ©∂•≤≥©¥π ض 4•∏°≥ -•§©£°¨ "≤°Æ£® °¥ '°¨∂•≥¥ØÆ #,).)#!, #/-0%4%.#% %8!-).!4)/. 2!4).' &/2- %&&' (%&&) 3©¥• 3¥µ§•Æ¥, '•Æ %£(§ %¥® #°≥• %∏°≠©Æ•≤ %∏, -3, 9≤(0§ ...... DIRECTIONS: Please rate the student on each item using the following scale. / = Honors 0 = High Pass 1 = Pass (Satisfactory) % = Pass (Marginal) 2 = Failing (Unsatisfactory) .! = Not Appropriate for case /¢≥•≤∂•§ §µ≤©Æß 0°¥©•Æ¥ %ƣصƥ•≤5 Performed focused history for patient’s problem(s)6 6 6 6 6 6 6 6 6 6 / 0 1 % 2 .! Elicited information about the patient’s family and support systems6 6 6 6 / 0 1 % 2 .! Performed focused physical exam for patient’s problem(s)6 6 6 6 6 6 6 / 0 1 % 2 .! Provided education appropriate to the problem(s)6 6 6 6 6 6 6 6 6 6 6 / 0 1 % 2 .! /¢≥•≤∂°¥©ØÆ≥ °Æ§ #Ø≠≠•Æ¥≥5 AY 06-07 2 /¢≥•≤∂•§ §µ≤©Æß 0°¥©•Æ¥ %ƣصƥ•≤5 Provided education related to health promotion. 6 6 6 6 6 6 / 0 1 % 2 .! Performed age and gender specific screening6 6 6 6 6 6 6 6 / 0 1 % 2 .! Utilized information in medical record for current problem(s) and for health risk assessment6 6 6 6 6 6 6 6 6 6 6 6 / 0 1 % 2 .! Involved patient in decision-making, including patient’s understanding of the problem(s76 6 6 6 6 6 6 6 6 6 6 6 6 / 0 1 % 2 .! Established rapport with patient6 6 6 6 6 6 6 6 6 6 6 / 0 1 % 2 .! Asked about and responded to patient’s feelings and concerns6 6 6 6 / 0 1 % 2 .! Used open and close-ended questions appropriately6 6 6 6 6 6 / 0 1 % 2 .! History had appropriate organization and flow (efficient non-redundant questioning, closed sections with summary, transitional statements to next section etc.). 6 6 6 6 6 6 6 6 6 6 6 6 / 0 1 % 2 .! Speech and language were appropriate and easily heard and understood6 6 / 0 1 % 2 .! /¢≥•≤∂°¥©ØÆ≥ °Æ§ #Ø≠≠•Æ¥≥5 ,°¢Ø≤°¥Ø≤π 2•±µ•≥¥≥5 Applied cost effective evidence-based clinical principles to the selection and use of laboratory and diagnostic procedures. 6 6 6 6 6 / 0 1 % 2 .! /¢≥•≤∂°¥©ØÆ≥ °Æ§ #Ø≠≠•Æ¥≥5 AY 06-07 3 /≤°¨ 0≤•≥•Æ¥°¥©ØÆ5
91 Described nature of complaint (location, duration, onset etc.). 6 6 6 / 0 1 % 2 .! Included pertinent information from past medical history, family history, ROS and psychosocial history. 6 6 6 6 6 6 / 0 1 % 2 .! Described pertinent positive and negative findings from the history and physical exam. 6 6 6 6 6 6 6 6 6 6 6 6 / 0 1 % 2 .! Presented a differential diagnosis incorporating patient’s risk factors, and prevalence and incidence of the problem(s). 6 6 6 6 6 / 0 1 % 2 .! Demonstrated knowledge of the presenting problem(s), including an understanding of the disease process. 6 6 6 6 6 6 6 / 0 1 % 2 .! Outline treatment options appropriate for the patient. 6 6 6 6 6 / 0 1 % 2 .! Integrate bio-psychosocial data, including economic status and culture, in treatment and management. 6 6 6 6 6 6 6 6 6 / 0 1 % 2 .! Described health risk factors for patient with family, cultural and individual variables in mind. 6 6 6 6 6 6 6 6 6 6 / 0 1 % 2 .! Described a plan to gather further data or begin appropriate intervention or management for the problem(s) and / or health promotion. 6 6 / 0 1 % 2 .! Presented data in well-organized and well-integrated manner. 6 6 6 / 0 1 % 2 .! /¢≥•≤∂°¥©ØÆ≥ °Æ§ #Ø≠≠•Æ¥≥5 AY 06-07 4 1µ•≥¥©ØÆ : !Æ≥∑ •≤5 Responses to question(s) concerning the application of ∞°¥®Ø∞®π≥©Ø¨Øßπ in understanding disease processes and the diagnosis of problem(s). 6 / 0 1 % 2 .! Responses to question(s) concerning the application of ∞®°≤≠°£Ø¨Øßπ in the treatment and management of problem(s). 6 6 6 6 6 / 0 1 % 2 .! Responses to question(s) concerning the use of and evidence for ©Æ¥•ß≤°¥©∂• ≠•§©£©Æ• for problem(s). 6 6 6 6 6 6 6 6 6 / 0 1 % 2 .! /¢≥•≤∂°¥©ØÆ≥ °Æ§ #Ø≠≠•Æ¥≥5 Summary: Please use the space below to make summary comments about this student’s performance. 3©ßÆ°¥µ≤• ض %∏°≠©Æ•≤5 ...... 2•∂©≥•§ *µ¨π %&&' ¢π ,-$ AY 06-07 5 Examiner’s Global Rating of Professionalism Please provide a summative global rating of the student’s professionalism in interactions with the patient and with you as the examiner. Failure to receive a satisfactory rating may result in failure of the case. Demonstrates respect, compassion, integrity, honesty; and willingly acknowledges omissions and deficiencies Lacks respect, 92 compassion, integrity, or honesty; insensitive to diversity; denies errors, omissions, or deficiencies Satisfactory Marginal Unsatisfactory Examiner’s Global Rating of Competence Please provide the following global ratings of the student’s performance. These ratings should be based on your observation of and interaction with the student during the exam and should include, but not be limited to, the performance already described by the exam rating form. These global ratings will ÆØ¥ be calculated as part of this student’s grade, but will provide a benchmark to be used in calibrating future scoring rubrics. Please provide a summative global rating of the student’s performance on the patient encounter portion of this case. Honors High Pass Pass Marginal Pass Fail Please provide a summative global rating of the student’s performance on the presentation and discussion portion of this case. Honors High Pass Pass Marginal Pass Fail Using your best judgment to combine the patient encounter with the presentation and discussion, please provide a summative global rating of the student’s performance on this case. Honors High Pass Pass Marginal Pass Fail
93
APPENDIX C
Preceptor Rating Form
94 Clerkship Evaluation Family Medicine 5 Clerkship Evaluation of John Doe School Year: 2007-2008 Period: 10 Evaluator: Layne Dearman Evaluator Capacity: test 5 Service: test 5 Start: 5/1/2008 End: 6/22/2008 1. History Taking Skills Consistently: Asks focused Hx, PMHx, and ROS questions in a logical sequence Identifies reason(s) for visit fully Explores sensitive information professionally Data is accurate/correct Utilizes medical record Updates medication list (including OTC and home remedies) and recent labs Skill: Focused history of patient's problem(s) 654321 654322 654323 654324 23456Did not observe Inconsistent performance of skills: History of common problems incomplete or inaccurate Misses key information or chronology unclear Does not ask about OTC or home remedies Awkard collection of sensitive information Does not utilize medical record Consistently: Elicits information from patient relating to family, support systems, stressors, culture, and socioeconomic factors Skill: Focused bio-psychosocial assessment
Comments: Comments Examples: Clerkship Evaluations Page 1 of 5 http://ardev.utmb.edu/ce/studentEval.asp?rec=1465 6/3/2008 2. Physical Examination Skills Consistently: Performs PE maneuvers correctly, including vital signs Able to perform focused exam for relevant area(s) Able to distinguish normal from abnormal findings Skill: Focused physical exam for patient's problem (s) 654321 654322 654323 654324 Inconsistent performance of skills: 23456Did not observe Does not perform PE manuever correctly Does not perform focused exam Misses abnormal findings Comments: Comments Examples: 3. Communication Skills Consistently: Presents data in appropriate, logical sequence without commentary 95 Uses proper medical terminology Focuses presentation to key information Establishes rapport with even the most difficult patients/families Changes and adapts communication style for individuals in distress, or with emotional impairment Uses appropriate language for patient/family understanding Adapts to changes in routine Tolerates interruptions and distractions without a loss of composure or productivity Skill: Communicates with faculty, patients, and other members of health care team 654321 654322 654323 654324 23456Did not observe Inconsistent performance of skills: Disorganized in presentation Missing important data Data inaccurate Does not use proper medical terminology Does not establish rapport with patient / family Not easily understood by patient / family Insentive to patient / family emotional state Difficulty adapting to changes in routine Difficulty in changing roles with other members of the health care team Doesn't tolerate distractions or interruptions Comments: Comments Examples: 4. Problem-Solving Skills Skill: Clinical reasoning skills for diagnosis of Clerkship Evaluations Page 2 of 5 http://ardev.utmb.edu/ce/studentEval.asp?rec=1465 6/3/2008 Consistently: Demonstrates thorough knowledge of common medical problems Appropriately interprets data to develop thorough, defensible assessments Able to problem-solve in a logical fashion Able to understand and interpret the important elements of Hx / Pe Appropriately prioritizes problems and DDx common problems 654321 654322 654323 654324 23456Did not observe Inconsistent performance of skills: Fund of knowledge spotty/sparse Has difficulty developing assessments with appropriately broad and defensible DDx Does not use a logical pattern to problem-solve Difficulty interpreting data Difficulty with prioritization of information Consistently: Demonstrates thorough knowledge of common medical problems Able to suggest appropriate diagnostic and therapeutic plan for level of training Demonstrates thorough knowledge of health maintenance and risk assessment Provides patient education for common problems and treatments appropriate to patients' understanding Demonstrates understanding and influence of biopsychosocial data by integrating appropriately into treatment and management decisions Demonstrates knowledge of / evidence for vitamins, supplements, and alternate approaches to health and 96 healing for treatment and management of common problems Skill: Management skills for common problems 654321 654322 654323 654324 23456Did not observe Inconsistent performance of skills: Spotty/sparse knowledge of broad treatment categories Sparse knowledge of Dx tools Excessive / unnecessary use of laboratory and diagnostic procedures Is not familiar with health maintenance and risk assessment guidelines Lacks knowledge needed to provide patient education for common problems at patient's level of understanding Fails to incorporate psychosocial data into treatment plan Fund of knowledge relating to vitamins, supplements, and alternative approaches to health and healing is spotty / sparse. Does not recognize potential contraindications for drug use with vitamins or supplements Consistently: Demonstrates thorough knowledge of drug classes and mechanisms of action Identifies drug side effects and contraindications for drug use Understands basic pathophysiology of common problems Uses basic science principles in problem-solving Skill: Application of basic sciences 654321 654322 654323 654324 23456Did not observe Inconsistent performance of skills: Fund of knowledge of drug classes used to treat common problems is spotty / sparse Does not apply basic science principles Does not understand basic pathophysiology of common problems Spotty / sparse knowledge of drug classes and mechanisms of action Difficulty recognizing side effects or contraindications of drug use Comments: Clerkship Evaluations Page 3 of 5 http://ardev.utmb.edu/ce/studentEval.asp?rec=1465 6/3/2008 Comments Examples: 5. Professionalism Consistently: Identifies learning needs Seeks out and benefits from feedback Demonstrates initiative to read materials related to patients' problems Utilizes technology (web-based or PDA) to research patients' problems Arrives prepared and on time Flexes schedule (arrives early / stays late) to maximize opportunities to learn Skill: Readiness to learn in clinical setting 654321 654322 654323 654324 23456Did not observe Inconsistent performance of skills: Appears unmovitated or disinterested in learning and/or improving skills 97 Waits to be told what to do Distracted by outside activities Chronically late or frequently requesting time away from activities in the clinic Lacks insight into learning needs Does not seem to benefit from feedback / constructive criticism Consistently: Demonstrates respect, compassion, integrity, and honesty Maintains professional appearance / attire at all times Considers needs of patients, families and colleagues Adheres to all rules and regulations relating to students as described in the Universities' Academic Policies and the Clerkship syllabus Skill: Professionalism 654321 654322 654323 654324 23456Did not observe Inconsistent performance of skills: Comments: Comments Examples: 6. DEAN'S LETTER COMMENTS Clerkship Evaluations Page 4 of 5 http://ardev.utmb.edu/ce/studentEval.asp?rec=1465 6/3/2008 UTMB | Search | Directories | Toolbox | News | Employment | Contact | Sitemap UT System | Reports to the State | Compact With Texans | Statewide Search This site published by Academic Computing/Academic Resources Last modified: May 23, 2008 02:58:27 PM -0500 Copyright © 2006 The University of Texas Medical Branch. Please review our privacy policy and Internet guidelines. 7. OVERALL PERFORMANCE Please check one statement as your assessment of the student's overall performance. Comments above should support your overall assessment. 65432Serious weaknesses noted in one or more areas. Student would clearly benefit from remediation. 65432Some weaknesses noted. Performance is below that expected for a student at this level; student might benefit from remediation. 65432Performance at expected level for training. Competence demonstrated in ALL skills areas necessary to pass clerkship objectives. Most students should fall in this category. 65432Performance above level of training in some areas. Excellence demonstrated in some skill areas, competency in all other areas. 65432 Performance consistently above that expected for this level. Excellence demonstrated in ALL skill areas. This category should be reserved for the top 10% of all students you have ever taught. Save to work on later Submit for review Clerkship Evaluations Page 5 of 5 http://ardev.utmb.edu/ce/studentEval.asp?rec=1465 6/3/2008
98
APPENDIX D
Self-Directed Learning Readiness Scale / Learning Preference Assessment
99 LEARNING PREFERENCE ASSESSMENT Instructions: This is a questionnaire designed to gather data on learning preferences and attitudes towards learning. After reading each item, please indicate the degree to which you feel that statement is true of you. Please read each choice carefully and choose/highlight the response which best expresses your feeling.
There is no time limit for the questionnaire. Try not to spend too much time on any one item; however, your first reaction to the question will usually be the most accurate. RESPONSES
y
y
a s l
i e t ' d r s w h r i
t n e
s a h o h l i
t
h t
e . d
h l
; t I e e I
e
f e
l ;
e
m . e I e . n f i m
e e t e e
;
I
f m f
h e
m e ; m o I f i i
e h w t m
t ; o t
e
s e f e m e f u e
e l r o h h f u
m t t a t
r
o
e m f t h
f s i f
l u
t l e o . . y r
r
n a a t u a e y y e
a h r h w a a v
s t
u h w t e
e t l r e n f w w
t u n
a n
a s
e o m s s y t t s y h t i i i r b l t s t s f e
l h h e l a
e o o o t t
e a
v r t l
l y y u m m m o e e e l o a s a ITEMS l o r e e f A S m a f N w U w A 1. I'm looking forward to learning as 1 2 3 4 5 long as I'm living. 2. I know what I want to learn 1 2 3 4 5 3. When I see something that I don't 1 2 3 4 5 understand, I stay away from it 4. If there is something I want to learn, 1 2 3 4 5 I can figure out a way to learn it. 5. I love to learn. 1 2 3 4 5 6. It takes me a while to get started on 1 2 3 4 5 new projects. 7. In a classroom, I expect the teacher 1 2 3 4 5 to tell all class members exactly what to do at all times. 8. I believe that thinking about who 1 2 3 4 5 you are, where you are, and where you are going should be a major part of every person's education. 9. I don't work very well on my own. 1 2 3 4 5 10. If I discover a need for information 1 2 3 4 5 that I don't have, I know where to go to get it. 11. I can learn things on my own better 1 2 3 4 5 100
y
y
a s l
i e t ' d r s w h r i
t n e
s a h o h l i
t
h t
e . d
h l
; t I e e I
e
f e
l ;
e
m . e I e . n f i m
e e t e e
;
I
f m f
h e
m e ; m o I f i i
e h w t m
t ; o t
e
s e f e m e f u e
e l r o h h f u
m t t a t
r
o
e m f t h
f s i f
l u
t l e o . . y r
r
n a a t u a e y y e
a h r h w a a v
s t
u h w t e
e t l r e n f w w
t u n
a n
a s
e o m s s y t t s y h t i i i r b l t s t s f e
l h h e l a
e o o o t t
e a
v r t l
l y y u m m m o e e e l o a s a ITEMS l o r e e f A S m N a f w U w A than most people. 12. Even if I have a great idea, I can't 1 2 3 4 5 seem to develop a plan for making it work. 13. In a learning experience, I prefer to 1 2 3 4 5 take part in deciding what will be learned and how. 14. Difficult study doesn't bother me if 1 2 3 4 5 I'm interested in something. 15. No one but me is truly responsible 1 2 3 4 5 for what I learn. 16. I can tell whether I'm learning well 1 2 3 4 5 or not. 17. There are so many things I want to 1 2 3 4 5 learn that I wish that there were more hours in a day. 18. If there is something I have decided 1 2 3 4 5 to learn, I can find time for it, no matter how busy I am. 19. Understanding what I read is a 1 2 3 4 5 problem for me. 20. If I don't learn, it's not my fault. 1 2 3 4 5 21. I know when I need to learn more 1 2 3 4 5 about something. 22. If I can understand something well 1 2 3 4 5 enough to get a good grade on a test, it doesn't bother me if I still have questions about it. 23. I think libraries are boring places. 1 2 3 4 5 24. The people I admire most are 1 2 3 4 5 always learning new things. 25. I can think of many different ways 1 2 3 4 5 101
y
y
a s l
i e t ' d r s w h r i
t n e
s a h o h l i
t
h t
e . d
h l
; t I e e I
e
f e
l ;
e
m . e I e . n f i m
e e t e e
;
I
f m f
h e
m e ; m o I f i i
e h w t m
t ; o t
e
s e f e m e f u e
e l r o h h f u
m t t a t
r
o
e m f t h
f s i f
l u
t l e o . . y r
r
n a a t u a e y y e
a h r h w a a v
s t
u h w t e
e t l r e n f w w
t u n
a n
a s
e o m s s y t t s y h t i i i r b l t s t s f e
l h h e l a
e o o o t t
e a
v r t l
l y y u m m m o e e e l o a s a ITEMS l o r e e f A S m N a f w U w A to learn about a new topic. 26. I try to relate what I am learning to 1 2 3 4 5 my long-term goals. 27. I am capable of learning for myself 1 2 3 4 5 almost anything I might need to know. 28. I really enjoy tracking down the 1 2 3 4 5 answer to a question. 29. I don't like dealing with questions 1 2 3 4 5 where there is not a right answer. 30. I have a lot of curiosity about 1 2 3 4 5 things. 31. I'll be glad when I'm finished 1 2 3 4 5 learning. 32. I'm not as interested in learning as 1 2 3 4 5 some other people seem to be. 33. I don't have any problem with basic 1 2 3 4 5 skills. 34. I like to try new things, even if I'm 1 2 3 4 5 not sure how they will turn out. 35. I don't like it when people who 1 2 3 4 5 really know what they're doing point out mistakes I am making. 36. I'm good at thinking of unusual 1 2 3 4 5 ways to do things. 37. I like to think about the future. 1 2 3 4 5 38. I'm better than most people are at 1 2 3 4 5 trying to find out the things I need to know. 39. I think of problems as challenges, 1 2 3 4 5 not stop signs. 40. I can make myself do what I think I 1 2 3 4 5 102
y
y
a s l
i e t ' d r s w h r i
t n e
s a h o h l i
t
h t
e . d
h l
; t I e e I
e
f e
l ;
e
m . e I e . n f i m
e e t e e
;
I
f m f
h e
m e ; m o I f i i
e h w t m
t ; o t
e
s e f e m e f u e
e l r o h h f u
m t t a t
r
o
e m f t h
f s i f
l u
t l e o . . y r
r
n a a t u a e y y e
a h r h w a a v
s t
u h w t e
e t l r e n f w w
t u n
a n
a s
e o m s s y t t s y h t i i i r b l t s t s f e
l h h e l a
e o o o t t
e a
v r t l
l y y u m m m o e e e l o a s a ITEMS l o r e e f A S m N a f w U w A should. 41. I'm happy with the way I 1 2 3 4 5 investigate problems. 42. I become a leader in group learning 1 2 3 4 5 situations. 43. I enjoy discussing ideas. 1 2 3 4 5 44. I don't like challenging learning 1 2 3 4 5 situations. 45. I have a strong desire to learn new 1 2 3 4 5 things. 46. The more I learn, the more exciting 1 2 3 4 5 the world becomes. 47. Learning is fun. 1 2 3 4 5 48. It's better to stick with the learning 1 2 3 4 5 methods that we know will work instead of always trying new ones. 49. I want to learn more so that I can 1 2 3 4 5 keep growing as a person. 50. I am responsible for my learning— 1 2 3 4 5 no one else is. 51. Learning how to learn is important 1 2 3 4 5 to me. 52. I will never be too old to learn new 1 2 3 4 5 things. 53. Constant learning is a bore. 1 2 3 4 5 54. Learning is a tool for life. 1 2 3 4 5 55. I learn several new things on my 1 2 3 4 5 own each year. 56. Learning doesn't make any 1 2 3 4 5 difference in my life. 57. I am an effective learner in the 1 2 3 4 5 classroom and on my own. 103
y
y
a s l
i e t ' d r s w h r i
t n e
s a h o h l i
t
h t
e . d
h l
; t I e e I
e
f e
l ;
e
m . e I e . n f i m
e e t e e
;
I
f m f
h e
m e ; m o I f i i
e h w t m
t ; o t
e
s e f e m e f u e
e l r o h h f u
m t t a t
r
o
e m f t h
f s i f
l u
t l e o . . y r
r
n a a t u a e y y e
a h r h w a a v
s t
u h w t e
e t l r e n f w w
t u n
a n
a s
e o m s s y t t s y h t i i i r b l t s t s f e
l h h e l a
e o o o t t
e a
v r t l
l y y u m m m o e e e l o a s a ITEMS l o r e e f A S m N a f w U w A 58. Learners are leaders. 1 2 3 4 5
© 1977, Dr. Lucy M. Guglielmino Electronic format © 2004, Drs. Paul J. and Lucy M. Guglielmino This instrument is copyrighted. The reproduction of any part of it by mimeograph, photostat, electronic, or in any other form, whether the reproductions are sold or furnished free for use, is a violation of copyright law.
104
APPENDIX E
How to Interpret Your SDLRS/LPA Score
105 HOW TO INTERPRET YOUR SDLRS/LPA SCORE
Your score is a measure of your current level of Self-Directed Learning Readiness.
If your score is between: Then your readiness for self-directed learning is:
58-176 Low 177-201 Below Average 202-226 Average 227-251 Above Average 227-252 High
Some people have a low level of readiness because they have consistently been exposed to other-directed instruction. The most important thing to remember about your score is that it can be changes. Most persons with low or average levels of self-directed learning readiness can increase their readiness with awareness and practice.
The average score for adults completing the questionnaire is 214. The standard deviation is 25.59. The SDLRS measures your readiness for self-directed learning. Research has suggested that individuals who have developed high self-directed learning skills tend to perform better in jobs requiring:
1. A high degree of problem solving ability. 2. A high degree of creativity. 3. A high degree of change.
Persons with high SDLRS scores usually prefer to determine their learning needs and plan and implement their own learning. This does not mean that they will never choose to be in a structured learning situation. They may well choose traditional courses or workshops as a part of a learning plan.
Persons with average SDLRS scores are more likely to be successful in more independent situations, but are not fully comfortable with handling the entire process of identifying their learning needs and planning and implementing their learning.
Persons with below average SDLRS scores usually prefer very structured learning options such as lecture and traditional classroom settings.
Again, your SDLRS score indicates your current level of readiness for self-directed learning. Research studies have proven that levels of SDL readiness can be raised through appropriate educational interventions.
106
REFERENCES
Abraham, R. R., Upadbya, S., & Ramnarayan, K. (2005). Self-directed learning.
Advances in Physiology Education, 29, 135-136.
Accreditation Council for Graduate Medical Education (2006). ACGME: Outcome
project. General competencies. Retrieved July 27, 2007, from
http://www.acgme.org/outcomecomp/compFull.asp#3
Ainoda, N., Onishi, H., & Yasuda, Y. (2005). Definitions and goals of “ self-directed
learning” in contemporary medical education literature. Annals of the Academy of
Medicine Singapore, 34, 515-519.
Albanese, M. (2000). Problem-based learning: Why curricula are likely to show little
effect on knowledge and clinical skills. Medical Education, 34, 729-738.
American Association of Medical Colleges (2007). Retrieved July 7, 2007, from
http://www.aamc.org/
American Board of Surgery In-Training Examination (2008). About us. Retrieved
November 11, 208, from http://absite.org/user1/aboutus1.php
Ananthakrishnan, N. (1993). Objective structured clinical/practical examination
(OSCE/OSPE). Journal of Postgraduate Medicine, 39, 82-84.
107 Ansell, J.S., Boughton, R., Cullen, T., Hodges, C., Nation, E., Peters, P., et al. (1979).
Lack of agreement between subjective ratings of instructors and objective testing
of knowledge acquisition in a urological continuing medical education course.
Journal of Urology, 122(6), 721-723.
Appel, J., Friedman, E., Fazio, S., Kimmel, J., & Whelan, A. (2002). Educational
assessment guidelines: A clerkship directors in Internal Medicine commentary.
Teacher Web, Inc.
Arroyo-Jimenez, M. D. M., Marcos, P., Martinez-Marcos, A., Artacho-Pérula, E.,
Blaizot, X., Muñoz, M., et al. (2005). Gross anatomy dissections and self-directed
learning in medicine. Clinical Anatomy, 18, 385-391.
Association of American Medical Colleges (1998). Report 1: Learning objectives for
medical student education. Retrieved October 31, 2008, from
https://services.aamc.org/Publications/index.cfm?fuseaction=Product.displayFor
m&prd_id=198&prv_id=239
Bandura, A. (1989). Human agency in social cognitive theory. American Psychologist,
44(9), 1175-1184.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.
Englewood Cliffs, NJ: Prentice Hall.
Bligh, J. G. (1992). Independent learning among general practice trainees: An initial
survey. Medical Education, 26, 497-502.
Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of
educational goals handbook I (Cognitive Domain). London: Longman Green.
108 Blumberg, P. (2000). Evaluating the evidence that problem-based learners are self-
directed learners: A review of the literature. In D. H. Evensen & C.E. Hmelo
(Eds.). Problem-based learning: A research perspective on interactions (pp. 199-
226). New Jersey: Lawrence Erlbaum Associates Publishers.
Boud, D. (1998). (Ed.). Developing student autonomy in learning. (London: Kogan
Page).
Box, B. J. (1982). Self-directed learning readiness of students and graduates of an
associate degree nursing program (Doctoral dissertation, Oklahoma State
University, 1982). Dissertation Abstracts International, 44, 679A
Bradley, P., Oterholt, C., Nordheim, L., & Bjørndal, A. (2005). Medical students’ and
tutors’ experiences of directed and self-directed learning programs in evidence-
based medicine. Evaluation Review, 29(2), 149-177.
Brockett, R. G. (1985). Methodological and substantive issues in the measurement of
self-directed learning readiness. Adult Education Quarterly, 36, 15-24.
Brockett, R., Durr, R., Guglielmino, L., Guglielmino, P., Liddell, T., Muller, K., et al.
(2008, January). What have we learned about self-directed learning through the
use of the Self-Directed Learning Readiness Scale? Paper presented at the
meeting of the International Self-Directed Learning Symposium, Cocoa Beach,
FL.
Brockett, R. G., & Hiemstra, R. (1991). Self direction in adult learning: Perspectives in
theory, research and practice. New York, NY: Routlege.
109 Bryan-Wunner, V.C. (1991). Verification of competencies needed by entry-level
recreation, park and leisure professionals in Florida and the relationship of
competency perception to readiness for self-directed learning. Ed.D. dissertation,
Florida Atlantic University, United States – Florida. Retrieved July 19, 2009 from
Dissertation & Thesis @ Florida Atlantic University – FCLA database
(Publication No. AAT 9123003.
Bulik, R. J. (2003). Correlating performance outcomes and self-directed learning. In H.
B. Long & Associates (Eds.), Current developments in e-learning and self-
directed learning [CD] (pp. 76-90). Boynton Beach, FL: Motorola University.
Bulik, R. J., Burdine, R. L., & Shokar, G. S. (2007). Seamless integration of self- and
other directed learning in web-based cases. International Journal of Self-Directed
Learning, 4(1), 1-15.
Caffarella, R., & Caffarella, P. (1986). Self-directedness and learning contracts in adult
education. Adult Education Quarterly, 36, 226-234.
Caffarella, R., & Merriam, S. B. (2000). Linking the individual learner to the context of
adult learning. In A.L. Wilson & E.R. Hayes (Eds.), Handbook of adult and
continuing education (pp. 55-70). San Francisco, CA: Jossey-Bass.
Candy, P. C. (1995). Physicians teach thyself [sic]: The place of self-directed learning in
continuing medical education. Journal of Continuing Education in Health
Professions, 15, 80-90.
Candy, P. C. (1991). Self-direction for lifelong learning. San Francisco, CA: Jossey-Bass.
110 Carraccio, C., & Englander, R. (2000). The objective structured clinical examination: A
step in the direction of competency-based evaluation. Archives of Pediatric and
Adolescent Medicine, 154, 736-741.
Champagain, M.I., Bhattacharya, N., Jain, B.K., Kaimi, K.R., Kiorala, S., &
Jayawickramasajah, P.T. (1998). Introducing problem-based learning into an
organ system programme. Medical Teacher, 20, 587-589.
Clark, J. A. K. (1991). Self directed learning skills and clinical performance: A
comparison of traditionally taught and learning contract-taught nursing students
(Doctoral dissertation, University of Southern Mississippi). Dissertation Abstracts
International, 51, 2236.
Cohen, R., Reznick, R. K., Taylor, B. R., Provan, J., & Rothman, A. (1990). Reliability
and validity of the Objective Structured Clinical Examination in assessing
surgical residents. American Journal of Surgery, 160, 302-305.
Colliver, J. (2000). Effectiveness of problem-based learning curricula. Academic
Medicine, 75, 259-266.
Craik, G. L. (1840). Pursuit of knowledge under difficulties: Its pleasures and rewards.
New York: Harper & Brothers.
D’ A Slevin, O., & Lavery, C. (1991). Self-directed learning and student supervision.
Nurse Education Today, 11, 368-277.
Danis, C., & Tremblay, N. (1988). Autodidactic learning experiences: Questioning
established adult learning principles. In H.B. Long & Associates, Self-directed
learning: Application and theory. Athens, GA: Adult Education Department,
University of Georgia.
111 Davis, D., O’ Brien, M. A., Preemantle, N., Wolf, F. M., Mazmanian, P., & Taylor-
Vaisey, A. (1999). Impact of formal continuing medical education: Do
conferences, workshops, rounds, and other traditional continuing education
activities change physician behavior or health care outcomes? Journal of the
American Medical Association, 282, 867-874.
Davis, D. A., Thompson, M. A., Oxman, A. D., & Haynes, R. B. (1999). Changing
physician performance: A systematic review of the effect of continuing medical
education strategies. Journal of the American Medical Association, 274, 700-705.
Delahaye, B. L., & Choy, S. (2000). The Learning Preference Assessment (Self-Directed
Learning Readiness Scale). In Maltby, J., Lewis, C.A., & Hill, A. Commissioned
reviews of 250 psychological tests. Edwin Mellen Press, Wales, U.K.
Derrick, G. M. (2003). Creating environments conducive for lifelong learning. New
Directions for Adult and Continuing Education, 100, 5-18.
Dixon, E. (1989). Nurse readiness and time spent in self-directed learning. Journal of
Continuing Education in Nursing, 22, 215-218.
Dolmans, D. H. J. M., DeGrave, W., Wolfhagen, I. H. A. P., & van der Vleuten, C. P. M.
(2005). Problem-based learning: Future challenges for educational practice and
research. Medical Education, 39, 732-741.
Dolmans, H.J.M., & Schmidt, H.G. (1994). What drives the student in problem-based
learning? Medical Education, 28: 372-380.
Donnon, T., Paolucci, E. O., & Violato, C. (2007). The predictive validity of the MCAT
for medical school performance and medical board licensing examinations: A
meta-analysis of the published research. Academic Medicine, 82(1), 100-106.
112 Dornan, T., Boshuizen, H., Cordingley, L., Hilder, S., Hadfield, J., & Scherpbier, A.
(2004). Evaluation of self-directed clinical education: Validation of an
instrument. Medical Education, 38, 670-678.
Dornan, T., Hadfield, J., Brown, M., Boshuizen, H., & Scherpbier, A. (2005). How can
medical students learn in a self-directed way in the clinical environment? Design-
based research. Medical Education, 39, 356-364.
Emblen, J.D., & Gray, G.T. (1990). Comparison of nurses’ self-directed learning
activities. The Journal of Continuing Education in Nursing, 21, 56-61.
Etienne, P., & Julian, E. R. (2001, November). Identifying behaviors of successful
medical students and residents. [Electronic Version]. Analysis in Brief, 1(4).
Retrieved July 20, 2008, from: http://www.aamc.org/data/aib/aibissues/
aibvol1_no4.pdf
Fang, D. (2004, June). An analysis of the relationship between medical students’
educational indebtedness and their careers in research. [Electronic Version].
Analysis in Brief, 4(1).
Finestone, P. (1984). A construct validation of the Self-Directed Learning Readiness
Scale with labor education participants. (Doctoral dissertation, University of
Toronto, 1984). Dissertation Abstracts International, 46, 5A.
Friedman, C.P., Gatti, G.G., Franz, T.M., Murphy, G.C., Wolf, F.M., Heckerling, P.S., et
al. (2005). Do physicians know when their diagnoses are correct? Implications for
decision support and error reduction. Journal of General Internal Medicine, 20,
334-339.
113 Garrison, G., Matthew, D., & Jones, R.F. (2007, May). Future medical school applicants,
part 1: Overall trends. [Electronic Version]. Analysis in Brief, 7(3).
Garrison, G., Mikesell, C., & Matthew, D. (2007, April). Medical school graduation and
attrition rates. [Electronic Version]. Analysis in Brief, 7(2).
Ghosh, S. (2007). Combination of didactic lectures and case-oriented problem-solving
tutorials toward better learning: Perceptions of students from a conventional and
medical curriculum. Advances in Physiology Education, 31, 193-197.
Goetzl, E. J., Cohen, P., Downing, E., Erat, K., & Jessiman, A. G. (1973). Quality of
diagnostic examinations in a university hospital outpatient clinic. Annals of
Internal Medicine, 78(4), 481-489.
Goleman, D., Boyatzis, R., & McKee, A. (2002). Primal leadership: Realizing the power
of emotional intelligence. Boston: Harvard Business School Press.
Graeve, E.A. (1987). Patterns of self-directed learning of registered nurses. (Doctoral
dissertation, University of Minnesota, 1987). Dissertation Abstracts International,
38, 6467A.
Guglielmino, L. M. (1978). Development of the self-directed learning readiness scale.
(Doctoral dissertation, University of Georgia, 1977). Dissertation Abstracts
International, 38, 6467A.
Guglielmino, L.M. (1996). An examination of self-directed learning readiness and
selected demographic variables of top female executives. In Long, H.B. &
Associates: Current developments in self-directed learning (pp. 11-22). Norman,
OK: Public Managers Center, University of Oklahoma.
114 Guglielmino, L. M., Asper, D. A, II, Findley, B. W., Lunceford, C. R., McVey, R. S.,
Payne, S. M. et al. (2005). Common barriers, interrupters and restarters in the
learning projects of highly self-directed adult learners. [Electronic Version].
International Journal of Self-Directed Learning, 2(1), 71-93.
Guglielmino, L. M., & Guglielmino, P. J. (1984). Guglielmino’ s Self-Directed Learning
Readiness Scale. Self-Directed Learning and Health Professionals, NETWORK,
2. 2.
Guglielmino, L.M., & Guglielmino, P.J. (1988). Self-directed learning in business and
industry. An information age imperative. In H.B. Long & Associates (Eds.), Self-
Directed learning: Application and theory (pp. 125-148). Athens, GA: Adult
Education Department, University of Georgia.
Guglielmino, L. M., & Guglielmino, P. J. (1991). Learning Preference Assessment
facilitator guide. King of Prussia, PA: Organization Design and Development.
Guglielmino, L. M., & Guglielmino, P. J. (2006). Culture, self-directed learner readiness
and per capita income in five countries. SAM Advanced Management Journal,
71(2), 21-28, 57.
Guglielmino, L.M., & Hillard, L. (2007). Promoting achievement: A case study of
exemplary elementary principals. ERS Spectrum, 25, 13-24.
Guglielmino, L. M., Long, H. B., & Hiemstra, R. (2004). Self-direction in learning in the
United States. International Journal of Self-Directed Learning, 1, 1-17.
115 Guglielmino, L. M., Mazmanian, P.E., Guglielmino, P.J., Hoban, J. D., & Pololi, L.
(2002). L’ aptitude a l’ apprentissage autodirige d’ etudiants en medecine: Une
analyse preliminaire des donnees de trios facultes de medecine aux Etats-Unis
(Self-directed learning readiness of medical students: A preliminary examination
of data from three medical schools in the United States). In Carre, P. & Moisan,
A. La formation autodirigee: Aspects psychologiques et pedagogiques. Paris,
France: L’ Harmattan.
Guglielmino, P. J., & Klatt, L. A. (1994). Self directed learning readiness as a
characteristic of the entrepreneur. In H.B. Long & Associates (Eds.), New ideas
about self-directed learning (pp.161-174). Norman, OK: Oklahoma Research
Center for Continuing Professional and Higher Education of the University of
Oklahoma.
Hall-Johnson, K. (1981). The relationship between readiness for self-directed learning
and participating in self-directed learning. (Doctoral dissertation, Iowa State
University, 1981). Dissertation Abstracts International, 46, 7A
Harden, R. M., Stevenson, M., Downie, W. W., & Wislon, G. M. (1975). Assessment of
clinical competence using objective structured examination. British Medical
Journal, 1, 447-451.
Harvey, B.J., Rothman, A.I., & Fecker, R.C. (2003). Effect of an undergraduate medical
curriculum on students’ self-directed learning. Academic Medicine, 78, 1259-
1265.
116 Hassan, A.J. (1981). An investigation of the learning projects of adults of high and low
readiness for self-directed learning. (Doctoral dissertation, Iowa State University,
1981). Dissertation Abstracts International, 42, 3838A-3839A.
Hiemstra, R. (2004). Self-directed learning lexicon. [Electronic Version]. International
Journal of Self-Directed Learning, 1(2), 1-6.
Hiemstra, R. (1999). Self-directed learning. In W.J. Rothwell & K.J. Sensenig (Eds.), The
Sourcebook for Self-Directed Learning (pp. 9-20). Amherst, MA: HRD Press.
Hiramanek, N. (2005). Self-directed learning and continuing medical education.
Australian Family Physician, 34(10), 879-880.
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn?
Educational Psychology Review, 16(3), 235-266.
Hoban, J. D., Lawson, S. R., Mazmanian, P. E., Best, A. M., & Seibel, H. R. (2005). The
Self-Directed Learning Readiness Scale: A factor analysis study. Medical
Educator, 39(4), 370-379.
Hodder, R. V., Rivington, R. N., Calcutt, L. E., & Hart, I. R. (1989). The effectiveness of
immediate feedback during the objective structured clinical examination. Medical
Education, 23, 184-188.
Holm, M. B. (1980). Occupational therapy selection criteria and educational processes:
An integrated model (Doctoral dissertation, University of Nebraska, 1980).
Dissertation Abstracts International, 41, 2856A.
Holy Bible: New international version (1978). East Brunswick, NJ: International Bible
Society.
117 Horn, K. D., Sholehvar, D., Nine, J., Gilbertson, J., Hatton, C., Richert, C., et al. (1997).
Continuing medical education on the World Wide Web (WWW). Interactive
pathology case studies on the Internet. Archives of Pathology and Laboratory
Medicine, 121, 641-645.
Hosmer, W. (1847). Self-education: Or the philosophy of mental improvement. Havana,
NY: W.H. Ongley.
Houle, C. O. (1961). The inquiring mind. Madison: University of Wisconsin Press.
Houle, C. O. (1988). The inquiring mind (2nd ed.). Norman, OK: Oklahoma Research
Center for Continuing Professional and Higher Education, University of
Oklahoma.
Hull, A. L., Hodder, S., Berger, B., Ginsberg, D., Lindheim, N., Quan, J., et al. (1995).
Validity of three clinical performance assessments of internal medicine clerks.
Academic Medicine, 70(6), 517-522.
Hutton, C. A. (1985). An evaluation of program design to increase the self-directed
learning readiness of registered nurses in a baccalaureate program. Paper
presented at Conference of R.N. Educators at the University of Buffalo, Buffalo,
NY.
Johnstone, J. W., & Rivera, R. (1965). Volunteers for learning: A study of the
educational pursuits of adults. Chicago: Aldine.
Jones, C.J. (1990). A study of the relationship of self-directed learning readiness to
observable behavior characteristics in an adult basic education program (Doctoral
dissertation, University of Georgia, 1989). Dissertation Abstracts International,
50, 3446.
118 Jowett, N., LeBlanc, V., Xeroulis, G., MacRae, H., & Dubrowski, A. (2007). Surgical
skill acquisition with self-directed practice using computer-based video training.
The American Journal of Surgery, 193(2), 237-242.
Karani, R., Leipzig, R. M., Callahan, E. H., & Thomas, D. C. (2004). An unfolding case
with a linked objective structured clinical examination (OSCE): A curriculum in
inpatient geriatric medicine. Journal of American Geriatric Society, 52, 1191-
1198.
Katherein, M. A. (1981). A study of self-directed continued professional learning of
members of the Illinois Nurses’ Association: Content and process. (Doctoral
dissertation, Northern Illinois University, 1981). Dissertation Abstracts
International, 42, 1092A.
Knowles, M. S. (1962). The adult education movement in the United States. NY: Holt,
Rinehart, and Winston, Inc.
Knowles, M. S. (1968). Andragogy, not pedagogy. Adult Leadership, 16(10), 350-352,
386.
Knowles, M. S. (1970). The modern practice of adult education: Andragogy versus
pedagogy. NY: Association Press.
Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers. NY:
Association Press.
Konje, J. C., Abrams, K. R., & Taylor, D. J. (2001). How discriminatory is the objective
structured clinical examination (OSCE) in the assessment of clinical competence
of medical students? Journal of Obstetrics & Gynecology, 21, 223-227.
119 Kulich, J. (1970). A historical overview of the adult self-learners. In Northwest Institute
Conference on Independent Study: the adult as a self-learner. University of
British Colombia, Vancouver, Vancouver, BC.
Lane, J. L., Ziv, A., & Boulet, J. B. (1999). A pediatric clinical skills assessment using
children as standardized patients. Archives of Pediatric and Adolescent Medicine,
153, 637-644.
Lengacher, C., & Van Cott, M. L. (1992). Nursing research related to educational re-
entry for the registered nurse: Review of research in nursing education. Nursing,
5, 75-106.
Leach, D. (2006). Evaluation of competence. An ACGME perspective. American Journal
of Physical Medicine and Rehabilitation, 79, 487-489.
Lerner, M. (1957). America as a civilization. NY: Simon & Schuster.
Linares, A. Z. (1987). Learning style preference, self-directed learning readiness and
locus of control of baccalaureate nursing students (Doctoral dissertation,
University of Houston, 1987). Dissertation Abstracts International, 49, 683.
Linares, A. Z. (1989). A comparative study of learning characteristics of RN and generic
students. Journal of Nursing Education, 28, 354-360.
Long, H. B., & Agyekum, S. K. (1984). Self-directed learning: Assessment and
validation. In H.B. Long & Associates (Eds.), Self-directed learning: Application
and theory (pp. 253-266). Lifelong Learning Research/Publication Project,
Department of Adult Education, University of Georgia.
Lujan, H. L., & DiCarlo, S. E. (2006). Too much teaching, not enough learning: What is
the solution? Advances in Physiology Education, 30, 17-22.
120 Malin, A. (1985). Curriculum planning for self-directed learning in nursing education
(Doctoral dissertation, University of New York at Buffalo). Dissertation
Abstracts International, 47, 5A.
Malloy, M. H., Perkowski, L., Callaway, M., & Speer, A. (1998). The relationship
between preceptor expectations and student performance on two pediatric
objective structured clinical examination stations. Archives of Pediatric &
Adolescent Medicine, 152, 806-811.
Maltby, J., Lewis, C. A., & Hill, A. (2000). The handbook of psychological tests. Wales,
UK: Edwin Mellen Press.
Mamary, E., & Charles, P. (2003). Promoting self-directed learning for continuing
medical education. Medical Teacher, 25(2), 188-190.
Matsell, D. G., Wolfish, N. M., & Hsu, E. (1991). Reliability and validity of the objective
structured clinical examination in paediatrics. Medical Education, 25, 293-299.
Maxim, B. R., & Dielman, T. E. (1987). Dimensionality, internal consistency, and
interrater reliability of clinical performance ratings. Medical Education, 21, 130-
137.
McCune, S. K., Guglielmino, L. M., & Garcia, G. (1990). Adult self-direction in
learning: a preliminary analytic investigation of research using the Self-Directed
Learning Readiness Scale. In H.B. Long & Associates, Advances in self-directed
learning research (pp. 147-154). Norman, OK: Oklahoma Research Center for
Continuing Professional and Higher Education.
121 McDonald, F. S., Zeger, S. L., & Kolars, J. C. (2007). Factors associated with medical
knowledge acquisition during internal medicine residency. Society of General
Internal Medicine, 22, 962-968.
McDonald, J. B. (1967). Independent learning: The theme of the conference. In G.
Gleason (Ed.). Theory and nature of independent learning: A symposium.
Scranton, PA: International Textbook Co.
Medical College Admission Test Web Site (1995-2009). Home. Retrieved June 11, 2007,
from http://www.aamc.org/students/mcat/
Medical Group Management Association (2005). Physician compensation and
production survey: 2005 Report based on 2004 data - #6398. Retrieved June 16,
2007, from http://www.mgma.com
Mennin, S. P., Friedman, M., Skipper, B., Kalishman, S., & Snyder, J. (1993).
Performance on the NBME I, II, and III by medical students in the problem-based
learning and conventional tracks at the University of New Mexico. Academic
Medicine, 68(8), 616-624.
Merriam, S. B., & Caffarella, R. S. (1999). Learning in adulthood: A comprehensive
guide, (2nd ed). San Francisco, CA: Jossey-Bass Publishers.
Merriam, S. B., Caffarella, R. S., & Baumgartner, L. (2007). Learning in adulthood (3rd
ed.). San Francisco: Jossey-Bass.
Mezirow, J. (2000). Learning as transformation. San Francisco, CA: Jossey-Bass.
Middlemiss, M.A. (1989). Relationship of self-directed learning readiness and job
characteristics to job satisfaction for professional nurses. Paper presented at
Research in Nursing Education 7th Annual Meeting, San Francisco, CA.
122 Miflin, B. (2004). Adult learning, self-directed learning and problem-based learning:
deconstructing the connections. Teaching in Higher Education, 9(1), 43-53.
Miller, S. H. (2005). American Board of Medical Specialties and repositioning for
excellence in lifelong learning: Maintenance of certification. Journal of
Continuing Education for Health Professionals, 25(3), 151-156.
Mocker, D., & Spear, G. (1982). Lifelong learning: Formal, nonformal, informal, and
self-directed. (Information Series No. 241). Columbus, OH: ERIC Clearinghouse
for Adult, Career, and Vocational Education, Ohio State University.
Moore, R. J. (1987). Predictors of success in courses for nurses requiring a degree of self-
direction (Doctoral dissertation, Florida Atlantic University, 1987). Dissertation
Abstracts International, 48, 1670.
Muller, Kenneth E. (2007). Emotional intelligence and self-directed learning. Ph.D.
dissertation, Florida Atlantic University, United States -- Florida. Retrieved June
11, 2009, from Dissertations & Theses @ Florida Atlantic University - FCLA
database. (Publication No. AAT 3266421).
Murray, J. A. (1987). The effect of a clinical internship on the self-directed learning
readiness of baccalaureate nursing students (Doctoral dissertation, The University
of Iowa). Dissertation Abstracts International, 49, 1036.
Musal, B., Gursel, Y., Taskiran, H.C., Ozan, S., & Tuna, A. (2004). Perceptions of first
and third year medical students on self-study and reporting processes of problem-
based learning. BMC Medical Education, 4, 16.
National Board of Medical Examiners Web Site (1996-2009). NBME home. Retrieved
July 14, 2008, from http://www.nbme. org
123 Nolan, J., & Nolan, M. (1997). Self-directed and student-centered learning in nurse
education, British Journal of Nursing, 6, 51-55.
Norman, G. R., & Schmidt, H. G. (1992). The psychological basis of problem-based
learning: A review of the evidence. Academic Medicine, 67, 557-565.
O’ Kell, S. P. (1988). A study of the relationship between learning style, readiness for
self-directed learning, and teaching preference of learner nurses in one health
district. Nurse Education Today, 8, 197-204.
O’ Shea, E. (2003). Self-directed learning in nurse education: A review of the literature.
Journal of Advanced Nursing, 43(1), 62-70.
Pazmino, W. R. (2001). God our teacher. Grand Rapids, MI: Baker Academic.
Pearson, M. A. (1989). Feedback of learning preferences information and its effects on
affective responses, independent study time, and self-directed learning readiness
of nurses in a self-directed hospital orientation program. Ph.D. dissertation,
Northwestern University, United States –Illinois. Retrieved February 26, 2009,
from Dissertations & Theses: Full Text database. (Publication No. AAT
9015402).
Petrusa, E. R., Blackwell, T. A., & Ainsworth, M. A. (1990). Reliability and validity of
an objective structured clinical examination for measuring clinical performance in
residents. Archives of Internal Medicine, 150, 573-577.
Pierre, R. B., Wierenga, A., Barton, M., Branday, J. M., & Christie, C. D. C. (2004).
Student evaluations of an OSCE in paediatrics at the University of the West
Indies, Jamaica. BMC Medical Education, 4.
124 Pilling-Cormick, J., & Bulik, R. (1999). A preliminary study exploring the use of the
Self-Directed Learning Perception Scale in a clinical setting. In H. B. Long &
Associates (Eds.), Contemporary ideas and practices in self-directed learning.
Norman, OK: University of Oklahoma.
Posner, F. G. (1989). A study of self-directed learning, perceived competence and
personal orientation among students in an open alternative high school (Doctoral
dissertation, University of Denver, 1989). Dissertation Abstracts International.
51, 03-A.
Qin, Z., Johnson, D. W., & Johnson, R. T. (1995). Cooperative versus competitive effects
and problem solving. Review of Educational Research, 65(2), 129-143.
Roberts, D. G. (1986). A study of the use of the Self-Directed Learning Readiness Scale
as related to selected organizational variables. Dissertation Abstracts
International, 47, 1218A.
Rugh, J.D., Goggins, T., & Hatch, J.P. (2009). Practitioners’ perceptions of half-life of
dental school knowledge. Presented at the American Association for Dental
Research Annual Conference, April, Miami, FL.
Rutala, P. J., Witzke, D.B., Leko, E. O., Fulginiti, J. V., & Taylor, P. J. (1991). Sharing
of information by students in an objective structured clinical examination.
Archives of Internal Medicine, 151, 541-544.
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S.
(1996). Evidence-based medicine: What it is and what it isn’ t. British Medical
Journal, 312, 71-72.
125 Savoie, M. (1979). Continuing education for nurses: Predictors of success in courses
requiring a degree of learner self-direction (Doctoral dissertation, University of
Toronto, 1979). Dissertation Abstracts International, 40, 12A.
Schmidt, H. G., Vermeulen, L., & Van Der Molen, H. T. (2006). Long term effects of
problem-based learning: a comparison of competencies acquired by graduates of a
problem-based and conventional medical school. Medical Education, 40, 562-
567.
Schwartz, R. W., Burgett, J. E., Blue, A. V., Donnelly, M. B., & Sloan, D. A. (1997).
Problem-based learning performance-based testing: Effective alternatives for
undergraduate surgical education assessment and student performance. Medical
Teacher, 19, 19-23.
Schwartz, R., Donnelly, M., Drake, D., & Sloan, D. (1993). Faculty sensitivity in
determining medical students’ clinical competence. Clinical Investigative
Medicine, 16(Supplement), B87.
Schwartz, R. W., Donnelly, M. B., Nash, P. P., & Young, B. (1992). The development of
critical congruency in a problem-based medical curriculum. Academic Medicine,
67, 694-696.
Schwartz, R., Donnelly, M., Sloan, D. A., Johnson, S. B., & Strodel, W.E. (1995). The
relationship between faculty ward evaluations OSCE, and ABSITE as measures of
surgical intern performance. American Journal of Surgery, 169(4), 414-417.
Scott, J. N., & Markert, R. J. (1994). Relationship between critical thinking skills and
success in preclinical courses. Academic Medicine, 69, 920-924.
126 Shokar, G. S., Shokar, N. K., Romero, C. M., & Bulik, R. J. (2002). Self-directed
learning: Looking at outcomes with medical students. Family Medicine, 34(3),
197-200.
Silver, B., & Hodgson, C. S. (1997). Evaluating GPAs and MCAT scores as predictors of
NBME I and clerkship performances based on students’ data from one institution.
Academic Medicine, 72(5), 394-396
Simon, S. R., Bui, A., Day, S., Berti, D., & Volkan, K. (2007). The relationship between
second-year medical students’ OSCE scores and USMLE Step 2 scores. Journal of
Evaluation in Clinical Practice, 13(6), 901-905.
Simon, S. R., Volkan, K., Hamann, C., Duffey, C., & Fletcher, S. W. (2002). The
relationship between second-year medical students’ OSCE scores and USMLE
Step 1 scores. Medical Technology, 24(5), 536-539.
Skaggs, R. (1981). The relationships between involvement of professional nurses in self-
directed learning activities, loci of control, and readiness for self-directed learning
measures (Doctoral dissertation, University of Texas at Austin, 1981).
Dissertation Abstracts International, 42, 1906A.
Slaughter, R. (2009, July). Experience in the Doctor of Pharmacy Program at Wayne
State University increases students' readiness for self-directed learning. Poster
session presented at the annual conference of the American Association of
Colleges of Pharmacy, Boston, MA.
Sloan, D.A., Donnelly, M.B., Schwartz, R.W., & Strodel, W.E. (1995). The objective
structured clinical examination: The new gold standard for evaluating
postgraduate clinical performance. Annals of Surgery, 222(6), 735-742.
127 Smiles, S. (1882). Self-help. London: John Murray.
Smith, L. J., Price, D. A., & Houston, I. B. (1984). Objective structured clinical
examination compared with other forms of student assessment. Archives of
Disease in Childhood, 59, 1173-1176.
Smith, M. K. (2002). Malcolm Knowles, informal adult education, self-direction and
andragogy. Encyclopedia of informal education. Retrieved July 19, 2008, from
www.infed.org/thinkers/et-knowl.htm
Southwestern Medical Center (2008). Family Medicine Clerkship – Helpful Hints –
Preparing for the Exam. Retrieved November 11, 2008, from
http://www.southwestern,edu/utsw/cda/dept30716/files/54571.html
Spear, G., & Mocker, D. (1984). The organizing circumstance: Environmental
determinants in self-directed learning. Adult Education Quarterly, 40, 3701A.
Steele, D. J., Medder, J. D., & Turner, P. (2000). A comparison of learning outcomes and
attitudes in student- versus faculty-led problem-based learning: An experimental
study. Medical Education, 34, 23-29.
Stipe, D.K. (1987). A self-directed method will enhance self-directedness in two-year
nursing students (Master’ s Thesis, California State University, 1987).
Dissertation Abstracts International, 26, 247.
Tärnvik, A. (2007). Revival of the case method: A way to retain student-centered
learning in a post-PBL era. Medical Teacher, 29 (1), e32-e36.
Tough, A. (1971). The adult’s learning projects: A fresh approach to theory and
practice in adult learning. Toronto: Ontario Institute for Studies in Adult
Education.
128 Tough, A. (1978). Major learning efforts: Recent research and future directions. Adult
Education, 28, 250-263.
Tough, A. (1979). The adult’s learning projects: A fresh approach to theory and
practice in adult learning. (2nd ed.). Toronto: Ontario Institute for Studies in
Adult Education
Tough, A. (1982). Intentional changes: A fresh approach to helping people change.
Chicago: Follett Publishing Company.
Trevena, L. J. (2007). Problem-based learning in public health workforce training: a
discussion of educational principles and evidence. NSW Public Health Bulletin,
18(1-2), 4-8.
United States Medical Licensing Examination. Retrieved June 11, 2007, from
http://www.usmle.org/default.asp
U. S. Department of Labor, Bureau of Labor Statistics (2007). Occupational Outlook
Handbook. Retrieved from http://www.bls.gov/oco/ocos074.htm on July 4, 2007.
UTMB Self-Study Summary (2006). University of Texas Medical Branch, Galveston, TX.
UTMB School of Medicine Bulletin, (2007-2009). University of Texas Medical Branch.
Retrieved July 4, 2007, from hhtp://wwws.som.utme.edu/
SOM%20Bulletin%20web.pdf
Violato, C., & Lockyer, J. (2006). Self and peer assessment of pediatricians, psychiatrists
and medical specialists. Advances in Health Sciences Education, 11, 235-244.
White, C. B. (2007). Smoothing out transitions: How pedagogy influences medical
students’ achievement of self-regulated learning goals. Advances in Health
Sciences Education, 12(3), 279-297.
129 Whittle, S. R., & Murdoch-Eaton, D. G. (2004). Lifelong learning skills: How
experienced are students when they enter medical school? Medical Teacher, 6,
576-578.
Whittle, S. R., & Murdoch-Eaton, D. G. (2001). Attitudes towards transferable skills in
medical undergraduates. Medical Education, 35, 148-153.
Wiley, K. (1981). Effects of a self-directed learning project and preference for structure
on self-directed learning readiness of baccalaureate nursing students. (Doctoral
dissertation, Northern Illinois University, 1981). Dissertation Abstracts
International, 43, 1A.
Williams, B. (2004). Self direction in a problem based learning program. Nurse
Education Today, 24, 277-285.
Williams, G. C., Saizow, R. B., & Ryan, R. M. (1999). The importance of self-
determination theory for medical education. Academic Medicine, 74(9), 992-995.
Wray, N. P., & Friedland, J. A. (1973). Detection and correction of house staff error in
physical diagnosis. Annals of Internal Medicine, 78, 481-489.
Yalcin, B. M., Karahan, T. F., Karadenizil, D., & Sahin, E. M. (2006). Short-term effects
of problem-based learning curriculum on students’ self-directed skills
development. Croatian Medical Journal, 47, 491-498.
Zsiga, Peter L. (2007). Self-directed learning readiness, strategic thinking and leader
effectiveness in directors of a national nonprofit organization. Ph.D. dissertation,
Florida Atlantic University, United States -- Florida. Retrieved June 11, 2009,
from Dissertations & Theses @ Florida Atlantic University - FCLA database.
(Publication No. AAT 3269548)
130