Promoting the Teaching of Critical Thinking Skills Through Faculty Development Linda S

Promoting the Teaching of Critical Thinking Skills Through Faculty Development Linda S. Behar-Horenstein, Ph.D.; Gail Schneider-Mitchell, R.D.H., M.P.H.; Randy Graff, Ph.D. Abstract: Practical and effective faculty development programs are vital to individual and institutional success. However, there is little evidence that program outcomes result in instructional changes. The purpose of this study was to determine if and how faculty development would enhance participants’ use of critical thinking skills in instruction. Seven faculty members from the University of Florida College of Dentistry and one faculty member from another health science college participated in six weekly two-hour faculty development sessions in spring 2007 that focused on enhancing critical thinking skills in instruction. Kaufman’s and Rachal’s principles of andragogy (adult learning) were used to design the sessions. Participants used learning journals to respond to four instructor-assigned prompts and provided one presentation to peers. With the use of qualitative methods, eight themes emerged across the learning journals: teaching goals, critical thinking, awareness of learners, planned instructional change, teaching efficacy, self-doubt, external challenges, and changes made. Five of eight participants incorporated critical thinking skills into their presentations at a mean level of 2.4 or higher on a 5-point scale using Paul and Elder’s behavioral definition of critical thinking skills. Faculty development opportunities that cause participants to reason through learning journals, peer presentations, and group discussion demonstrated the incorporation of critical thinking concepts in 63 percent of this cohort group’s presentations, suggesting that if evidence-based pedagogies are followed, instructional changes can result from faculty development. Dr. Behar-Horenstein is Distinguished Teaching Scholar and Professor, Department of Educational Administration and Policy, and Affiliate Professor, Department of Operative Dentistry, College of Dentistry, University of Florida; Ms. Schneider-Mitchell is Associate in Dentistry and Director of Curriculum and Instruction, College of Dentistry, University of Florida; and Dr. Graff is Assistant Director of Education and Training, Health Affairs-Academic Information Systems Information Support, University of Florida. Direct correspondence and requests for reprints to Dr. Linda S. Behar-Horenstein, Department of Educational Adminis- tration and Policy, University of Florida, P.O. Box 117049, 1212 Norman Hall, Gainesville, FL 32611-7049; 352-392-2391, ext. 299 phone; 352-846-2697 fax; [email protected]. Key words: faculty development, critical thinking, teacher behaviors, andragogy, adult learning, qualitative research Submitted for publication 11/11/08; accepted 4/1/09

ncreasingly, faculty development has become an providing them with feedback about their own per- important component in professional education. formance.2 IUsed to manage change and develop professional In an era in which dental educators are expe- skills, one of the primary purposes of faculty devel- riencing numerous fundamental changes in the cur- opment has been to improve instructional practice. riculum, new teaching and learning methods,3 and From the novice to the veteran professor in academia, rapid clinical and technological advances, teaching the practice of educating students in a clinical care has come to be recognized as a skill associated with profession requires lifelong learning. but separate from content expertise.4 Therefore, the The research question we asked in this study design, methodology, and implementation of practi- was this: How will faculty development focused on cal and effective faculty development programs are enhancing critical thinking skills influence partici- vital to the success of individual and institutional pants’ instructional practices? The purpose of this teaching effectiveness. The evidence-based char- study was to explore whether faculty development acteristics associated with program effectiveness in would foster the participants’ ability to infuse critical faculty development include the following: thinking skills into their instruction and to describe • experiential learning, applying what has been the evidence that showed their integration of critical learned and receiving feedback; thinking strategies. The development and assessment • systematic and constructive feedback to improve of this seminar were linked with concepts of evalu- performance; ation in andragogy, “the art and science of helping • peer feedback and collegial support to promote adults learn.”1 In this study, evaluation was viewed change; as an instructional methodology that focused on • adherence to teaching and learning principles of the learners’ knowledge base and experience, while adult (e.g., Steinart et al.5) and experiential learn-

June 2009 ■ Journal of Dental Education 665 ing (e.g., Kolb6) as an organizing structure; and • demonstration of the use of active learning meth- • multiple instructional methods to accommodate ods, such as case analysis and discussion, critical learning styles and to meet diverse objectives.7 appraisal of scientific evidence in combination with clinical application and patient factors, and structured sessions in which faculty and students Critical Thinking, the reason aloud about patient care.8 Critical thinking is also regarded as intellec- Learning Process, and tually engaged, skillful, and responsible thinking that facilitates good judgment because it requires Andragogy the application of assumptions, knowledge, and Current dental education accreditation stan- competence and the ability to challenge one’s own dards call for evidence of students’ critical think- thinking. Unlike other forms of thinking, critical ing in the learning process.8 This position requires thinking requires the use of self-correction and faculty competence in teaching students the devel- monitoring to judge the reasonableness of thinking opment of critical thinking skills, yet researchers as well as reflexivity. When using critical thinking, debate whether critical thinking can be learned or individuals step back and reflect on the quality of is a developmental process, regulated by motiva- that thinking. Simpson and Courtney pointed out that tions, dispositions, and personality traits. Despite critical thinking processes require active argumenta- differences of opinion, many researchers agree tion, initiative, reasoning, envisioning, and analyzing that critical thinking is “purposeful, self-regulatory complex alternatives and making contingency-related judgment which results in interpretation, analysis, value judgments.10 evaluation, and inference, as well as explanation of According to Banning, critical thinking in- the evidential, conceptual, methodological, crite- volves scrutinizing, differentiating, and appraising riological, or contextual considerations upon which information, as well as reflecting on information to judgment is based.”9 make judgments that will inform clinical decisions.11 The Commission on Dental Accreditation Brookfield asserted that identifying and challenging (CODA) recently proposed new standards for teach- assumptions as well as analyzing assumptions for ing predoctoral students critical thinking skills. Stan- validity are essential to critical thinking skills. He dard 2-9 stipulates that graduates must demonstrate also suggested that because critical thinkers possess competence in the use of critical thinking as it relates curiosity and skepticism, they are more likely to be to comprehensive patient care. To ensure that students motivated to provide solutions that resolve contra- develop this competence, the dental education cur- dictions.12 riculum must use pedagogical methods that provide Critical thinkers use these skills appropriately evidence of its development such as and usually without prompting. They are generally • explicit discussion of the meaning, importance, predisposed to think critically and to evaluate the and application of critical thinking; outcome of their thought processes.13 These attri- • use of questions by instructors that require students butes are important for health care practitioners. to analyze problem etiology, compare and evaluate Moreover, instructors’ incorporation of them into the alternative approaches, provide rationale for plans learning process must be intentional, not assumed of action, and predict outcomes; or random. Helping health professions educators • prospective simulations in which students perform learn to do this requires attention to the ways adult decision making; students learn. • retrospective critiques of cases in which decisions The principles and characteristics of “andra- are reviewed to identify errors, reasons for errors, gogy” were used to design the faculty development and exemplary performance; workshop opportunities at the University of Florida • assignments that require students to analyze prob- College of Dentistry in this study. Andragogy, a lems and discuss alternative theories about etiol- term rejuvenated by Malcolm Knowles, is “the art ogy and solutions, as well as to defend decisions and science of helping adults learn,” according to made; Rachal,14 and is the most persistent practice-based • asking students to analyze and discuss work prod- instructional method in adult education. Knowles ucts to compare how outcomes correspond to best proposed seven guidelines for teaching adults to be- evidence or other professional standards; and come self-directed and independent in their pursuit of

666 Journal of Dental Education ■ Volume 73, Number 6 new knowledge.1 At the heart of these guidelines is an adult students.16 Rubrics are used to articulate the approach based on collaboration between the facilita- required elements of a successful program. In our tor and the learner involving a learning contract that study, Paul and Elder’s rubric for defining critical includes learning objectives, strategies and resources, thinking17 was used to help participants understand evidence of achievement, and means for evaluation. the linkages between learning objectives and desired Additionally, voluntary participation is one premise outcomes. of andragogy, according to Knowles. These, coupled with a nonthreatening learning environment, are key to encouraging success of the individual learner. Methods Additional characteristics of andragogy can be found in work by Forrest and Peterson in which they A diverse, international group of seven faculty looked at the adult learner as an independent adapt- members from the College of Dentistry and one fac- able individual.15 Forrest and Peterson proposed four ulty member from an affiliated health science college assumptions regarding adult learners: adult learners participated in this study. The group consisted of five are self-directed, rely upon experience, are ready to females and three males; there was one participant learn, and have a performance-centered orientation each born in Taiwan, Brazil, Italy, England, Israel, to learning. Rachal extended Knowles’s guidelines and India and two born in the United States. Faculty for developing in-class activities and out-of-class members were invited to attend the faculty develop- assignments for faculty development seminars.14 ment seminar through email and announcements Rachal revised Knowles’s concept of “pedagogy about faculty development offerings. Participation in to andragogy” as a continuum and proposed seven the seminar was strictly voluntary and was subsidized criteria that reach towards the andragogy side of the by an educational foundation grant and departmen- continuum: tal funds. Faculty received no release time for their 1. Voluntary participation. participation. Approval to conduct the study was 2. Adult status: “learners who have assumed the obtained from the university’s Institutional Review social and culturally defined roles characteristic Board (IRB; UFIRB #2006-U-0791). Inquiry for of adulthood and who perceive themselves to be this study was carried out in accordance with IRB adult.” approval. Each participant signed a letter of informed 3. Collaboratively determined objectives: the consent during the first session. learner plays a significant, if not primary, role The faculty development seminar was designed in determining learning objectives. using the principles of andragogy (Table 1). The 4. Performance-based assessment of achievement: seminar consisted of six two-hour sessions in spring low-threat, performance assessment in which 2007: three were instructor-led sessions on the topics performance is either successful or not and in of teaching efficacy, curriculum auditing, and peer which the reason for pursuing the learning activ- observation; the other sessions focused on participant ity is to exhibit a specific desired outcome. presentations, including a discipline-based teaching 5. Measuring satisfaction: the inherent pleasure or session and oral recitation of participants’ changes in satisfaction of participating in a learning activ- planning, delivering, and evaluating teaching. After ity. each of the first three and the fifth sessions, the par- 6. Appropriate adult environment: physical and ticipants were asked to respond in their learning jour- psychological characteristics including creature nals to instructor-assigned prompts (Table 2). During comforts and a room arrangement that fosters a the fourth and fifth sessions, participants presented a sense of collaboration. Respect for the learner discipline-based lesson plan to showcase the kinds of as an adult, respect for the learner’s experiences, changes in teaching critical thinking skills they had minimization of anxiety, and avoidance of the articulated in their learning journals. Each participant “schooling experience.” 7. Technical issues: use of one or two facilitators received feedback about his or her presentation from in various settings.14 colleagues and the course codirectors. During the Faculty development programs that incorporate these last session, they read the contents of their personal principles can create a more meaningful experience statement of growth to the class. for the participants. Two types of data were collected and analyzed When writing about andragogy and assessment, to answer the research questions: learning journals Bolton suggested that rubrics are helpful to use with and a discipline-based teaching presentation. Par-

June 2009 ■ Journal of Dental Education 667 ticipants wrote four learning journals in response students write and reflect upon course content and to instructor prompts. Researchers do not agree on how it relates to their own experiences.18 Despite the a single definition of learning journals, but they do lack of an agreed-upon definition, researchers agree agree that learning journals share a common element: that learning journals promote active learning.

Table 1. Application of andragogy principles to the faculty development seminar Principle Application of Principle

1. Create comfortable and safe learning environments Participants met in a large conference room from 11 am where learners feel able to speak freely.1 to 1:00 pm over six sessions. Food and beverages were 2. Establish a climate conducive to adult learning.2 provided. Handouts were provided. 3. Provide a learning environment that is appropriate for A check-in was conducted at the beginning of each ses- adults.3 sion. The instructor took participants’ questions throughout each session. The course director responded to questions at the time they were asked. Learning journals served as a medium for participants to communicate with the course director. The course director read the journals and provided written feedback on them.

4. Learners and instructors select the use of methods and Participants developed a brief PowerPoint presentation to content.1 showcase the kinds of changes in teaching critical thinking 5. Create an organizational structure for participative skills they had articulated in their learning journals. learning.2

6. Learners diagnose their own needs.1 Participants were asked to document in learning journals 7. Diagnose learning needs.2 the areas of teaching in which they felt efficacious and those in which they did not, e.g., planning for instruction, delivering instruction, writing assessments, assigning grades, providing varied forms of instruction, managing students, and dealing with disruptive and/or confrontational students.

8. Learners write their own lesson objectives.1 Participants presented a lesson plan (with learning objec- 9. Learners formulate direction of learning (objectives).2 tives) within the PowerPoint presentation to showcase the 10. Learners collaboratively determine objectives.3 kinds of changes in teaching critical thinking skills that they had articulated in their learning journals.

11. Learners locate and use resources to achieve Participants developed discipline-based lesson plans. objectives.1 12. Learners develop design of activities.2

13. Instructors help learners achieve goals.1 Following each PowerPoint presentation, participants 14. Operate the activities.2 received feedback from the instructor and their peers 15. Technical issues.3 that identified aspects of the lesson that promoted critical thinking skills as well as what else (instructionally) the presentation could have done to promote critical thinking skills in the context of this lesson.

16. Learners evaluate success of their learning.1 Participants were asked to critique and document in their 17. Learners rediagnose (evaluate) learning needs.2 journals the learning activities in their course syllabi and 18. Learners participate in performance-based assessment identify a) those that currently promoted/required criti- of achievement.3 cal thinking skills, b) those they thought approximated students’ understanding and use of critical thinking skills, and c) those they wanted to promote/require critical thinking skills.

Sources: 1. Kaufman DM. ABC of learning and teaching in medicine: applying educational theory in practice. Br Med J 2003;326(7382):213–6. 2. Knowles MS. The modern practice of adult education: Cambridge adult education. Englewood Cliffs, NJ: Prentice-Hall, 1980. 3. Rachal JR. Andragogy’s detectives: a critique of the present and a proposal for the future. Adult Educ Q 2002;52(3):210–27.

668 Journal of Dental Education ■ Volume 73, Number 6 Table 2. Faculty development seminar prompts by session Session Prompt

1 Identify areas of teaching in which you feel efficacious and those in which you do not, e.g., planning for instruction, delivering instruction, writing assessments, assigning grades, providing varied forms of instruction, managing students, and dealing with disruptive and/or confrontational students. Using Paul and Elder’s critical thinking skill guide, critique the learning activities in your course syllabus and identify a) those you currently provide that promote/require critical thinking skills, b) those you think you approximate (almost do), and c) next steps for implementing/developing others that will deepen students’ understanding and use of critical thinking skills. 2 Construct a teaching philosophy based on the assigned readings and your previous teaching experiences. 3 View peer observation/evaluation scenarios. Write a one-minute paper to describe what you saw. Identify what aspects of the first/second peer observation could lead to improving instruction. Submit a one-minute paper and peer observation scenario feedback. Develop a PowerPoint presentation on a learning experience that requires students to demonstrate critical thinking skills or that approximates students’ use of critical thinking skills, or teaching an activity that is new for you that you believe will deepen students’ understanding and use of critical thinking skills. 4 Observe critical thinking skills presentation. Identify aspects of lesson that promote critical thinking skills. Identify what else (instructionally) could be done to promote critical thinking skills in the context of this lesson. 5 Observe critical thinking skills presentation. Identify aspects of lesson that promote critical thinking skills. Identify what else (instructionally) could be done to promote critical thinking skills in the context of this lesson. Write your personal statement of growth. 6 Present your personal statement of growth.

Learning journals provide a place in which were able to see how the scope of physical therapy participants can document self-reflections about practice assists this population.24 their experiences, beliefs, and responses to instruc- In our study, the learning journal provided a tor-provided prompts.19 Some researchers consider medium for written participant-instructor commu- learning journals essential to professional practice,20 nication. The first and second authors hand-coded especially when they are used to promote participant the dataset, and the third author analyzed the learn- insight into “self-awareness and learning.”21 Nurses ing journals, peer observation, and feedback using and educators have used learning journals “to en- NVivo, a qualitative analysis software that allows hance creative and critical thinking among students the researcher to import, sort, and analyze files and in the classroom.”21 Learning journals also give stu- can be used to code themes embedded in transcripts. dents an opportunity to step back, carefully articulate Our collective analysis produced several overlapping the meaning of new experiences, and consider how themes. As a result of this overlap, we developed a those experiences relate to their current experience. codebook of themes (Table 3), which consists of Kitchenham used reflective journals to help teachers eight cover terms and conceptual definitions. After self-evaluate transformative experiences as they inte- developing the codebook, each author independently grated educational technology into their classrooms,22 recoded the data. The criterion for coding segments and Brown reinforced the importance of reflection of the data was its fit within the conceptual definition with the use of experiential learning portfolios.23 As of a cover term. written documents, learning journals provide instruc- Participants’ presentations were analyzed us- tors with feedback concerning what students are un- ing Paul and Elder’s17 criteria for critical thinking derstanding, what connections they are making, and skills behaviors. To ensure that we were systematic how well they are progressing. While working with in assessing each of these behaviors, we reviewed physical therapy students, for example, Williams and the participants’ PowerPoint presentations together, Wessel found that students experienced positive shifts evaluated each participant’s demonstration of critical in their attitudes about working with the elderly and thinking skills (CTS) behaviors, and reached agree-

June 2009 ■ Journal of Dental Education 669 Table 3. Codebook for analyzing data Theme Definition

Awareness of learners Emergent knowledge of students as learners and how instructional practices influence the quality of learning and outcomes Changes made Modifications that occurred in instructional practices as a result of the course Critical thinking Ways in which faculty members promote and assess students’ ability to critically think External challenges Difficulties faculty members face during instruction and/or assessment Planned instructional changes Steps or plans with the intention to improve teaching Teaching efficacy Faculty perceptions of their relative strengths and abilities to teach Teaching goals Ways in which faculty members will generally enhance teaching to improve student learning Self-doubt Faculty perceptions of their weaknesses or perceptions that they cannot teach in ways they want

ment as to what constituted CTS. Each author Themes in Participants’ Learning evaluated the participants’ PowerPoint presentations independently by responding “yes” or “no” to these Journals criteria: Awareness of learners. This theme accounted 1. raises vital questions and problems, formulating for 13.4 percent (n=34) of the dataset. Chaim, one them clearly and precisely; of the participants, acknowledged becoming aware 2. gathers and assesses relevant information, using of students’ learning needs when he realized that he abstract ideas to interpret it effectively; did not need to cover all the information during in- 3. comes to well-reasoned conclusions and solu- struction and that he could hold students responsible tions, testing them against relevant criteria and for the content. Laura, another participant, reported standards; that she observed students’ reactions while teaching 4. thinks open-mindedly within alternative systems and modified her teaching methodology as needed. of thought, recognizing and assessing, as need Ilka identified the powerfulness of theory to practice be, their assumptions, implications, and practical connections. “By presenting clinical cases and ad- consequences; and ditional problem sets in both labs and lectures,” she 5. communicates effectively with others in figuring said, “I believe that students were more engaged in the 25 out solutions to complex problems. subject matter.” Adrianna reported that as she learned One point was awarded for a “yes” response, more about students, she came to an understanding and zero was awarded for a “no.” For each criterion, that teaching did not have to be dogmatic. a participant could earn a range from zero to three Changes made. This theme accounted for 7.1 points; thus, the overall score for each participant percent (n=18) of the dataset. Eric described how the could range from zero to fifteen. A mean score use of the automated response system (ARS) was across all of our responses was calculated for each great for “providing immediate feedback and guid- participant. ance.” Chaim described the changes he made to slide shows by adding a slide listing the objectives. He said he also started placing more questions on slides “as Results [a] break to the lecture and [to] serve as reminders The themes that emerged from the learning to the student [as an] application of the concept.” journals were awareness of learners, changes made, Adrianna wrote that she made several changes in her critical thinking, external challenges, planned in- instructional techniques during clinical instruction, structional changes, teaching efficacy, teaching goals, such as asking her students to name three things they and self-doubt. had learned that day. She explained that she encour-

670 Journal of Dental Education ■ Volume 73, Number 6 aged the pediatric dentistry residents to take greater confirmation, he planned to ask them what kinds of ownership of their learning by introducing varied question they had asked themselves. forms of instruction in her craniofacial anomalies External challenges. This theme accounted course. “Initially I found this concept quite daunting,” for 11.4 percent (n=29) of the dataset. Ilka reported she said, “but . . . [it] gave me the opportunity to ap- that being able to cover material and still have time preciate their initiative [and] a shared responsibility for class discussion was something new for her. She for learning. In addition, I introduced . . . case-based shared that “expecting [students] to do it so you can learning.” Lucy explained that she began asking stu- have a meaningful conversation and motivating them dents questions to make them think and talk aloud to comply presented a shift in my approach.” Adri- about the patient’s treatment more frequently than anna shared that she struggled with students’ low ex- she had before the seminar. pectations because she believed that some seemed to Critical thinking. This theme accounted for prefer a “quick fix” or accelerated pathway to getting 14.6 percent (n=37) of the dataset. Laura wrote that through dental school and did not seem to give much an excellent way to help students become critical thought to “the quality of the person or practitioner thinkers was to have them “compare multiple points on the other side of graduation.” She expressed her of view and understand how and why they differ.” concern “that the current culture in our dental school She also stated that she wanted students to make appears to pander to these low expectations, and comparisons using multiple sources. Lucy discussed subsequently I find myself competing in an unofficial two ways that she helps students to develop critical popularity contest.” Walter described how, while in thinking skills: first, “questioning students before the clinic, he struggled with himself when he tried [start-check], during, and at the end of each clinic to give out information rather than helping students session, making them access what they have learned arrive at the answers themselves. He reported that, in the classroom to apply in that session,” and second, after the seminar, he recognized what he was doing “highlighting/making them identify important points and, given this realization, had the ability to change. on each patient’s medical history, their anatomy, and Moreover, he shared that “there is a voice in my head specific traits in personality/habits that would influ- that won’t go away . . . continually asking me if I’m ence treatment plan and procedures.” connecting with students in an effective way.” Adrianna described how she used the probing Planned instructional changes. This theme technique to get students to use critical thinking. accounted for 13.4 percent (n=34) of the dataset. One of the questions she reported typically asking While describing the seminar assignments, Eric was “can you tell me which radiographs you would stated that it was during this course he had begun to recommend for this patient and the reason for your undertake his own efforts to improve his teaching recommendations?” She said she asked students to and that “peer evaluation led me to see styles which think about what they would do in their own private I will adopt for my teaching.” Ilka shared that she practice if the child’s parents, expected to pay for ad- planned to use a microphone to try and improve her ditional radiographs, questioned this because x-rays voice projection in large rooms. Marion planned to were taken six months ago; a typical question she challenge her students by asking questions at the asked was “can you tell me how you would justify beginning of the class because it would provide her your recommendation to a parent who is concerned with some information about what they know. She about cost and x-ray exposure?” explained that “if we revisit the question at the end Chaim described how he believed he could of the session . . . they may retain this information promote students’ development of critical thinking more.” Laura shared that when she writes questions skills. For example, in the first reading assignment in the future, she will be more careful and ask ques- of his course, he highlighted places where students tions that cannot immediately be answered. She would need to process the information and suggested planned to ask the type of questions that require a a couple of questions they should ask themselves. series of progressive responses before reaching the During the second reading assignment, he highlighted final answer. She also suggested that she might ask the area where they should process the information students to write their answers because “this will but did not provide any hints about questions. During give some time for everyone to think.” Lucy wrote the third and subsequent assignments, he let students that she was now thinking about what she wanted work on their own. If students came to him later for her students to learn during her lectures and how she

June 2009 ■ Journal of Dental Education 671 could get them thinking about the situations during reported, “I still have not got[ten] to the point I want her presentations. in terms of teaching and stimulating critical thinking Teaching efficacy. This theme accounted for . . . but I will get there!” Adrianna confided that she 11.8 percent (n=30) of the dataset. Walter expressed held pessimistic views of students and feared losing his growing sense of mastery in grading: “I think that I control over unruly or disruptive students: “I do not am improving with grading.” Marion reported that she yet feel equipped or confident to deal with students “felt quite efficacious in the way that [she] plan[ned] who are confrontational either in the clinic or class, for instruction.” Laura reported feeling effective in and I fear that it has in the past interfered with my her ability to help students think critically by “crafting delivery of instruction in these arenas.” questions that stimulate thinking . . . providing intel- These eight themes appeared in this rank lectual stimulation . . . [and] creating challenges [to] order from most to least frequent: teaching goals, make students think about alternative explanations.” critical thinking, awareness of learners, and planned Teaching goals. This theme accounted for 16.5 instructional changes, followed by self-doubt, teach- percent (n=42) of the dataset. Ilka described her aspi- ing efficacy, external challenges, and changes made rations for students. One of her teaching goals was to (Table 4). provide learning experiences in which students could explore science and discover their own interests. Ratings of Participants’ PowerPoint Some of Laura’s teaching goals were to “help students transition . . . from a passive to a reactive state . . . Presentations and from a reactive to an active state.” Lucy wrote We also rated the degree to which the seminar that her teaching goals included getting the point participants infused CTS into their presentations across; presenting the topic and relating it to clini- using Paul and Elder’s criteria as shown in Table 5. cal applications so that students would understand Their scores ranged from zero to 4.66; the average why what they were learning was important; giving was 2.41. Of the eight presentations, five participants students a reason for learning particular information; showed means of 2.4 or higher across four or five of and demonstrating to students how they would use or the five criteria, indicating that they had integrated need a particular concept or idea in dental practice. critical thinking skills across multiple criteria. Two of She wrote that she wanted to help students apply their the participants showed a lower integration of critical knowledge because she felt that this was motivating thinking skills with scores of 1.00 and .66 within two and engaged them during learning. or three of the rubric categories. The remaining par- Chaim, who teaches dental materials, wrote that ticipant did not demonstrate the use of critical thinking he wanted students to come up with reasons rather skills in her presentation and had a score of zero. than to remember reasons. He explained that, for students to demonstrate reasoning, they first would need to understand the scientific principles that Discussion govern how a material behaves in certain conditions. Overall, the majority of the faculty participants’ Adrianna shared that her personal teaching goal was journal citations were in the categories of teaching to be recognized as an educator inside and outside goals (16.5 percent), critical thinking (14.6 percent), the classroom and that she wanted students to learn awareness of learners, and planned instructional from their interactions in both settings. She noted changes (both at 13.4 percent). Given that the focus that “the role of a teacher is no longer restricted to of this seminar was on the teaching of critical think- the classroom.” ing, it is not surprising that the participants discussed Self-doubt. This theme accounted for 11.8 teaching goals and critical thinking most frequently. percent (n=30) of the dataset. Walter expressed his Of interest is that, within and across each theme, the self-doubt about planning and organizing material. participants discussed new insights in exploring sci- Marion expressed her doubt about using instruction, entific concepts, seeking alternative explanations, and saying, “I do not feel efficacious in changing my form questioning students as opposed to solely delivering of instruction during the didactic course.” Along the content. Some faculty members openly described their same lines, Lucy reported that prior to the faculty self-doubts, while others reflected on how external development seminar, she did not recognize that there challenges influenced the changes they had made and were so many ways to establish critical thinking and were planning to make in their teaching styles. that she had never recognized its importance. She

672 Journal of Dental Education ■ Volume 73, Number 6 Five of the eight participants utilized critical thinking skills with a Table 4. Themes by frequency (n=254) and sources (# of participants) score of 2.4 or higher across multiple Themes Frequency Sources criteria during their PowerPoint presentations. This finding indicates that Teaching goals 16.5% (n=42) 6 Critical thinking 14.6% (n=37) 8 they were able to raise vital questions Awareness of learners 13.4% (n=34) 8 clearly, assess relevant information, Planned instructional changes 13.4% (n=34) 7 come to well-reasoned conclusions Self-doubt 11.8% (n=30) 6 while testing them against criteria Teaching efficacy 11.8% (n=30) 8 and standards, and communicate ef- External challenges 11.4% (n=29) 6 fectively in figuring out solutions to Changes made 7.1% (n=18) 6 complex problems. A majority (63 percent) of the participants demonstrated incorporation of critical thinking skills. This participants’ learning journals and through teaching finding also demonstrates the powerfulness of effec- presentations to their peers. The participants’ willing- tive faculty development when it integrates experi- ness to try to assimilate new ways of thinking about ential learning and a diversity of teaching methods teaching, to take a chance, and to make themselves within one course.5 However, all of the participants vulnerable in front of their peers was also seen as experienced this teaching seminar as a developmental risk-taking behavior. As corroborated by the skill process. They reported feelings of self-efficacy and levels shown in their PowerPoint presentations, cog- self-doubt throughout the course. These findings are nitive change does not always result in immediate consistent with the challenges inherent in making behavioral change. It is important to point out that changes during a faculty development process in epistemic demand alone may not be enough to change general and during learning in particular. educators’ views of critical thinking: while they may In this study, the participants engaged actively write about it in journals, that does not mean they are in the learning process by sharing how they were able to synthesize related concepts and apply them responding to what they were learning and by describ- readily to their teaching.26 Furthermore, thinking ing how they might utilize that information in their and writing about critical thinking are not enough. teaching. Active integration of the material to varying Educators must be able to connect related concepts degrees of growth was evidenced by the content of to methods of teaching, while bearing in mind that

Table 5. Ratings of participants’ critical thinking skills during their PowerPoint presentations using Paul and Elder’s criteria Thinks open- mindedly within alternative systems of thought, Comes to recognizing Communicates Raises vital Gathers and well-reasoned and assessing, effectively with questions and assesses relevant conclusions and as need be, others in problems, information, solutions, testing their assumptions, figuring out Mean formulating using abstract them against implications, solutions to score them clearly ideas to interpret relevant criteria and practical complex across Participants and precisely it effectively and standards consequences problems criteria

Adrianna 0 0 0 0 0 0 Chaim 3 3 3 3 2 4.66 Eric 3 2 2 1 0 2.66 Ilka 2 2 2 2 2 3.33 Laura 3 3 3 1 2 4.00 Lucy 1 1 1 0 0 1.00 Marion 0 1 1 0 0 .66 Walter 3 2 3 0 1 3.00 Source for criteria: Paul R, Elder L. A miniature guide to critical thinking: concepts and tools. 4th ed. Dillion Beach, CA: Foundation for Critical Thinking Press, 2006:4.

June 2009 ■ Journal of Dental Education 673 changes in practice must also be reflected on within competent health care providers in a dynamically the larger picture of institutional commitment to changing health care field places new requirements teaching critical thinking and how this practice may on the educational process. Research has shown that affect student outcomes. relying solely on the use of lecturing is a relatively Possible limitations of the study were the small ineffective technique. Learning by doing not only sample size, the absence of a presession presentation, results in better retention and understanding of ma- and a lack of transferability of the findings beyond terial; it increases self-confidence, communication the context in which the study took place. skills, and the ability to engage in partnerships. In addition, if the revised CODA standards are approved, a substantial change in the way that dental schools Conclusions teach and assess critical thinking skills will be required.8 In the seminar described in our study, the The need for teaching higher-order or critical participants’ activities were aligned with examples thinking continues to grow as national testing has of CTS in the CODA standards (see Table 6). repeatedly demonstrated deficiencies in reading that Educators’ lack of formal training in teaching 27 requires reasoning and inference. These skills are has significant consequences for students who look essential to the demonstration of critical thinking. to their instructors as respected leaders whose be- Dental educators recognize that the status quo in haviors in thinking, speaking, and writing are those how to teach is being challenged because educating

Table 6. Alignment of participant learning experiences and CODA standards’ examples of critical thinking skill (CTS) activities CODA Standards Participants’ Learning Experiences Examples of Critical Thinking Skill Activities

Presentation and discussion in the seminar included Explicit discussion of the meaning, importance, and explicit discussion of what CTS is, what CTS is not, application of critical thinking. and what it looks like during teaching. Participants critiqued the learning activities in their Use of questions by instructors that require students course syllabus and identified those a) where they to analyze problem etiology, compare and evaluate currently promote/require critical thinking skills, alternative approaches, provide rationale for plans of b) where they approximate (almost do) the use of CTS, action, and predict outcomes. and c) where they would like to try and implement the use of CTS. Feedback regarding teaching methods provided Prospective simulations in which students perform participants with information about those areas where decision making. they wanted to or could have an opportunity to enhance their teaching of critical thinking skills and this required active learning and decision making. Participants identified in writing what aspects of the Retrospective critiques of cases in which decisions are first/second peer observation could lead to improving reviewed to identify errors, reasons for errors, and exemplary instruction. performance. Participants viewed peer observation/evaluation Writing assignments that require students to analyze scenarios and wrote one-minute papers to describe problems and discuss alternative theories about etiology what they saw. Additionally, participants identified in and solutions, as well as to defend decisions made. writing what aspects of the first/second peer observation could lead to improving instruction. Participants provided written feedback to colleagues. Asking students to analyze and discuss work products to They commented upon how well the presentations compare how outcomes correspond to best evidence demonstrated CTS and what other ways CTS could or other professional standards. be improved. Presentations represented an active learning method Demonstration of the use of active learning methods, such that required participants’ synthesis of understanding as case analysis and discussion, critical appraisal of scientific of CTS and utilization of it in a presentation of an evidence in combination with clinical application and patient existing lesson in which they previously had not factors, and structured sessions in which faculty and students used CTS. reason aloud about patient care.

674 Journal of Dental Education ■ Volume 73, Number 6 that they can model.28 Without a place to learn and 12. Brookfield S. Developing critical thinkers: challenging model these skills, educators unwittingly contribute adults to explore alternative ways of thinking and acting. to the pre-existing problem: lack of development of San Francisco: Jossey-Bass, 1987. 13. Halpern DF. Teaching critical thinking for transfer across students’ critical thinking skills. However, when fac- domains. Am Psychol 1998;53:449–55. ulty development is grounded in a theory (andragogy 14. Rachal JR. Andragogy’s detectives: a critique of the in this case1) that involves making meaning through present and a proposal for the future. Adult Educ Q experience and reflection29,30 and occurs during par- 2002;52(3):210–27. ticipation in communities of practice,31,32 it can result 15. Forrest SP III, Peterson T. It’s called andragogy. Acad Management Learning Educ 2006;5(1):113–22. in instructional practice changes beyond the seminar. 16. Bolton FC. Rubrics and adult learners: andragogy and Further research should explore the sustainability of assessment. Assessment 2006;18(3):5. these practices in the classroom and clinical learn- 17. Paul R, Elder L. A miniature guide to critical thinking: ing environments and other implications for faculty concepts and tools. 4th ed. Dillion Beach, CA: Foundation development.5 for Critical Thinking Press, 2006:4. 18. Varner D, Peck SR. Learning from learning journals: the benefits and challenges of using learning journal assign- REFERENCES ments. J Management 2003;27(1):52–77. 1. Knowles MS. The modern practice of adult education: 19. Behar-Horenstein LS, Schneider-Mitchell G, Graff R. Fac- Cambridge adult education. Englewood Cliffs, NJ: Pren- ulty perceptions of a professional development seminar. J tice-Hall, 1980. Dent Educ 2008;72(4):472–83. 2. Lowther-Berman RR. The evaluation of andragogy and 20. Plack M, Driscoll M, Bissett S, McKenna R, Plack T. A pedagogy as higher education instructional methodologies method for assessing reflective journal writing. J Allied in medication administration training for postsecondary, Health 2005;34(4):199–208. unlicensed residential behavioral health personnel in West 21. Thorpe K. Reflective learning journals: from concept to Virginia. Unpublished Ed.D. dissertation, The George practice. Reflective Practice 2004;5(3):327–43. Washington University, 2004. 22. Kitchenham A. Teachers and technology: a transformative 3. Fidler DC, Khakoo R, Miller LA. Teaching scholars pro- journey. J Transformative Educ 2006;4(3):202–25. grams: faculty development for educators in the health 23. Brown JO. Know thyself: the impact of portfolio develop- professions. Acad Psychiatry 2007;31(6):472–8. ment on adult learning. Adult Educ Q 2002;52(3):228– 4. Wilkerson L, Irby DM. Strategies for improving teaching 45. practices: a comprehensive approach to faculty develop- 24. Williams R, Wessel J. Reflective journal writing to obtain ment. Acad Med 1998;73(4):387–96. student feedback about their learning during the study 5. Steinart Y, Mann K, Centeno A, Dolmans D, Spencer J, of chronic musculoskeletal conditions. J Allied Health Gelula M, Prideaux D. A systematic review of faculty 2004;33(1):17–23. development initiatives designed to improve teaching 25. Paul RW, Nosich GM. A model for the national assessment effectiveness in medical education. BEME guide no. 8. of higher order thinking. In: Paul RW, ed. Critical think- Med Teacher 2006;28(6):497–526. ing. Santa Rosa, CA: Foundation for Critical Thinking, 6. Kolb DA. Experiential learning: experience as a source of 1992:78–123. learning and development. Englewood Cliffs, NJ: Prentice 26. Bell RL, Blair LM, Crawford BA, Lederman NG. Just do Hall, 1984. it? Impact of a science apprenticeship program on high 7. Kaufman DM. ABC of learning and teaching in medi- school students’ understandings of the nature of science cine: applying educational theory in practice. Br Med J and scientific inquiry.J Res Sci Teaching 2003;40(5):487– 2003;326(7382):213–6. 509. 8. Commission on Dental Accreditation. Standard 2-9 in 27. National Assessment of Educational Progress. NAEP information management and critical thinking. In: Ac- results show little change in reading skills. Reading Today creditation standards for dental education programs. 1990;7(1):8. Chicago: American Dental Association, 2008:19–20. 28. Brookfield SD. Assessing critical thinking. New Directions 9. Facione PA. Critical thinking: a statement of expert Adult Continuing Educ 1997;75:17–29. consensus for purposes of educational assessment and 29. Schon DA. Educating the reflective practitioner: toward a instruction. Millbrae, CA: The California Academic Press, new design for teaching and learning in the professions. 1990:2. San Francisco, CA: Jossey-Bass, 1987. 10. Simpson E, Courtney M. Critical thinking in nursing 30. Moon J. Reflection in learning and professional develop- education: literature review. Int J Nurs Practice 2002;8:89– ment: theory and practice. London: Kogan Page, 1999. 98. 31. Lave J, Wenger E. Situated learning: legitimate peripheral 11. Banning M. Measures that can be used to instill criti- participation. Cambridge, England: Cambridge University cal thinking in nurse prescribers. Nurse Educ Practice Press, 1991. 2006;6:98–105. 32. Boud D, Middleton H. Learning from others at work: communities of practice and informal learning. J Workplace Learning 2003;15(5):194–202.

June 2009 ■ Journal of Dental Education 675