Peer Review of Teaching at BYU: Guidelines and Suggestions Pertaining to the Rank and Status Process

Why peer review?

Quality teaching that enhances student learning is an important expectation of all faculty who teach at BYU. The University uses two primary evaluation tools to assess the quality of teaching: Student Ratings and Peer Review. Students are asked to provide feedback on the teaching/learning process from their perspective as learners. Peers are asked to assess teaching methods, materials, and outcomes from their perspective as teachers and members of an academic discipline.

There are two distinct purposes for peer review of teaching. The first is to provide formative (developmental) feedback to faculty members to help them improve their teaching. The second purpose is to provide a summative (accountability) evaluation for making personnel decisions (i.e., rank and status). The same criteria should be used for both formative and summative evaluations, even though the use of data is different. In both cases the evaluation criteria should be generally known and widely accepted within a department. (See Chism, Peer Review of Teaching, ch. 4)

When should peer reviews be conducted?

Summative peer reviews must be conducted in conjunction with the third- and sixth-year reviews for junior faculty. Within the context of the rank and advancement process, formative evaluations should help junior faculty improve their courses and prepare for summative evaluations. Therefore, they are most useful when conducted during the 1st or 2nd and the 4th or 5th years. It is recommended that a given peer review should not try to incorporate both formative and summative purposes.

Who conducts peer reviews?

Option A: If a department has organized a committee to conduct peer reviews (e.g., a Learning, Teaching and Curriculum Committee), then members of this committee could conduct both formative and summative reviews.

Option B: If a department does not use a standing committee for peer reviews, then a mentor could be assigned to conduct the formative review and an ad hoc group of faculty (2-3) could be assigned to conduct the summative review.

How should a summative peer review be conducted?

Section 3.3.2 of the University Rank and Status Policy provides the following direction.

Peer evaluation is as important for teaching as it is for scholarship. The department review committee will obtain at least two substantive confidential peer evaluations of teaching from BYU faculty members qualified to make evaluations of the faculty member's approach to pedagogy, teaching activities and materials. The faculty member will assemble a teaching portfolio containing syllabi, textbooks, handouts, multimedia materials, assignments, learning exercises, examinations, and other course materials. The peer evaluations should concentrate on a review of the teaching portfolio, but should also include classroom visits. Ideally, the classroom visits should be conducted over several semesters prior to the faculty member's third- and sixth-year reviews. Peer evaluations might best assess such areas as:

1 1. Whether the course reflects the current state of the discipline. 2. The faculty member's mastery of the course content. 3. The course objectives, including whether the course meets the objectives of the curriculum of which it is a part. 4. The course organization. 5. The methods used to foster and measure learning. 6. The materials in the teaching portfolio (syllabi, textbooks, handouts, multimedia materials, assignments, learning exercises, examinations, and other course materials). 7. The faculty member's general concern for and interest in teaching. 8. The overall quality of teaching.

Other examples of meaningful peer evaluation of teaching might include reports from graduate schools or employers regarding students' performance, and professional invitations based on a faculty member's reputation as a teacher.

What is the department chair’s role?

1. Facilitate ongoing conversations within the department regarding a) the importance of high quality teaching and learning, b) the value of peer review in facilitating continuous improvement (pre- and post-continuing status), and c) the appropriate criteria for guiding summative peer reviews. 2. Make peer review assignments and provide reviewers with all information and tools necessary to properly complete their assignments. 3. Ensure that the evaluation process is being conducted in a timely manner. A recommended timetable has been included in Appendix A. 4. Check the peer review reports for completeness and include them in the candidate’s promotion file.

What resource materials are available to guide the peer review process?

To assist faculty members assigned to conduct peer reviews of teaching, the Faculty Center has prepared the following supplemental materials. These should be treated as suggestions/recommendations based on study of the relevant literature. The establishment of standards or expectations beyond those laid out in the rank and status policy is the purview of colleges and departments.

A copy of the following book should be available in every department. It is the best source book on this subject. It contains a wide range of practical suggestions as well as useful peer-review forms.

Peer Review of Teaching: A Sourcebook, Nancy Van Note Chism (Bolton, MA: Anker Publishing Company), 1999.

We also recommend the following articles available from the Faculty Center:

Report of University Committee for the Study of Peer Evaluation (1999). (Proposals from a faculty committee.)

Theall, Michael (2002). “ Guidelines for Good Evaluation Practice .” Focus on Faculty, 10, 2-3 (part of “Student Ratings: Myth vs. Research Evidence”). (Discusses elements of a good evaluation system, especially how peer reviews and student ratings are complementary components of assessment.)

2 Sorenson, D. Lynn (1995). “ Beware the N of One ,” Focus on Faculty, 3, 6. (This short article warns of the danger of using oneself (an “n of one”) as the measure of good teaching when observing colleagues’ classes. The article also summarizes current research benchmarks of effective teaching as better criteria than the “n of one.”)

Cunningham, Michael (2001). “Observations Beyond the Lecture: New Learning Models Require New Evaluation Methods.” Teaching Professor, 15, 6. (Generally insightful suggestions)

Are there peer review tools we can use?

Experience has shown that the systematic application of standardized criteria significantly enhances the quality of a peer review process. To stimulate conversations within departments regarding the evaluation criteria and methods, the Faculty Center has provided some sample evaluation tools in Appendix B. Included in these materials are review forms to help assess a) course design and b) classroom instruction.

What should be included in the summative peer review report?

The rank and status policy stipulates that a candidate’s teaching portfolio need not be included in the materials sent forward to the college and university rank and advancement committees. Higher levels of the review process will rely on a thorough and comprehensive peer review report provided by the department. In light of this expectation, a recommended outline for this report is shown in Appendix C.

Appendix A: Recommended Timetable for Conducting Peer Reviews Appendix B: Sample Evaluation Tools Appendix C: Outline for Summative Peer Review-of-Teaching Report

3 APPENDIX A: RECOMMENDED TIMETABLE FOR CONDUCTING PEER REVIEWS. (The Third Year Review is used for illustrative purposes)

Year Semester/Month/Date Task Faculty member’s By the end of the Make sure the new faculty member has a mentor and that the first year second semester mentor provides formative feedback on teaching to the new faculty member (after reviewing materials and making classroom visits). Faculty member’s Fall Semester The Review Committee should arrange for at least two second year classroom visits during this year for summative purposes. The Department Review Committee should have in place the process by which faculty candidates for rank and status will be evaluated for their teaching. The Review Committee should arrange for a formal formative evaluation. Faculty member’s By [specify a month] The Review Committee should arrange for at least two third year classroom visits during this year for summative purposes. By [specify a month] The Department Review Committee should ask the faculty candidate to prepare a teaching portfolio. By [specify a date] The faculty candidate should submit the teaching portfolio to the Review Committee. By [specify a date] The Review Committee should arrange for at least two classroom visits during this year for summative purposes. By [specify a date] The Review Committee should have completed its review of the Teaching Portfolio, including all past evaluations based on classroom observations. By [specify a date] The Review Committee should complete its peer review report to be inserted into the candidate's dossier.

4 APPENDIX B: SAMPLE EVALUATION TOOLS

Notes:

1. These forms can be used for both summative and formative reviews. The major difference is that for summative reviews the comments section should be used to list specific examples or rationale supporting the rater's scores, whereas the comments section for formative reviews should be used to record constructive observations and suggestions for improvement.

2. These forms are provided as examples—their use is not required. Two different types of forms/approaches are included: rating forms and guidelines for a narrative report. Peer Review of Teaching (Chism, 1999) contains other examples of specific tools and evaluation criteria. Departments are encouraged to adapt these examples so that they will reflect what the department generally considers to be effective teaching leading to student learning, as well as department needs and circumstances.

Forms included:

1. Rating Form for Reviewing Course Design 2. Narrative Form for Reviewing Course Design 3. Rating Form for Reviewing Classroom Instruction 4. Narrative Form for Reviewing Classroom Instruction

5 RATING FORM FOR REVIEWING COURSE DESIGN (Adapted from Chism, 1999)

Course: ______Teacher: ______Reviewer: ______

A. Course Content Course syllabus, textbook, readings, assignments, handouts and other relevant course materials

After examining the above materials, please answer the following questions: Needs Satisfactory Exemplary To what extent . . . Improvement 1. Does the course content reflect the current state of the ______discipline? 2. Is the course content appropriate for this level (e.g., student ______preparation, sequence in the curriculum)? 3. Is the course content accurate? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements):

B. Teaching Materials Course syllabus, textbook, readings, assignments, handouts, multi-media and other relevant course materials

After examining the above materials, please answer the following questions: Needs Satisfactory Exemplary To what extent… Improvement 1. Are the teaching materials of high quality? ______2. Are the teaching materials and the course organized in ways that foster ______effective learning? 3. Is all relevant information provided and clearly presented? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements)

C. Instructional Design: Learning Goals Syllabus and other relevant course materials

After examining the above materials, please answer the following questions:

6 Needs Satisfactory Exemplary To what extent… Improvement 1. Are the goals for student learning clear? ______2. Do these goals focus on the most valuable/important learning

outcomes for students in this course? ______3. Do course learning goals reflect other relevant goals (e.g., BYU Aims,

program/department goals, discipline/professional standards)? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements)

D. Instructional Design: Learning Activities Syllabus, assignments, project descriptions, multi-media materials, learning exercises and other relevant course materials

After examining the above materials, please answer the following questions: Needs Satisfactory Exemplary To what extent… Improvement 1. Do the learning activities foster student achievement of the course

learning goals? ______2. Do the learning activities engage students in the learning process? ______3. Are students given ample opportunity to practice desired skills and

understanding prior to final assessments? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements)

E. Instructional Design: Learning Assessments Syllabus, exams, quizzes, projects, performances, and other course assignments

After examining the above materials, please answer the following questions: Needs Satisfactory Exemplary To what extent… Improvement 1. Are the assessments good measures of the learning goals for the course? ______2. Are the assessments aligned with the course learning activities? ______3. Are the performance expectations for course assessments clearly ______

7 communicated to students? 4. Do exams ask for more than comprehension and recall (e.g., application,

analysis, evaluation, creative work)? ______5. Is the level of performance expected on assessments appropriate for this

course (i.e., not too easy; not to difficult)? ______6. Are grading criteria for exams, projects, performances, and other

assignments clearly specified and followed? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements)

8 SAMPLE NARRATIVE OUTLINE FOR REVIEWING COURSE MATERIALS (Adapted from Chism, 1999)

Course: ______Teacher: ______Reviewer: ______

The candidate’s teaching portfolio should include the following materials for each course selected for evaluation: course syllabus, textbooks, handouts, multimedia materials, assignments, learning exercises, examinations, and other relevant course materials. The narrative summarizing the review of these materials should address the following areas:

A. Course Content To what extent do these materials show that the instructor has a broad, deep, and current knowledge of the content he or she is teaching? • Does the course content reflect the current state of the discipline? • Is the course content appropriate for this level (e.g., student preparation, sequence in the curriculum)? • Is the course content accurate?

B. Teaching Materials Are the teaching materials likely to enhance student learning? • Are the teaching materials of high quality? • Are the teaching materials and the course organized in ways that foster effective learning? • Is all relevant information provided and clearly presented?

C. Instructional Design To what extent does the portfolio show that the instructor uses good design principles to facilitate learning? 1. Learning Goals • Are the goals for student learning clear? • Do these goals focus on the most valuable/important learning outcomes for students in this course? • Do course learning goals reflect other relevant goals (e.g., BYU Aims, program/department goals, discipline/professional standards)? 2. Learning Activities • Do the learning activities foster student achievement of the course learning goals? • Do the learning activities engage students in the learning process? • Are students given ample opportunity to practice desired skills and understanding prior to final assessments? 3. Learning Assessments: • Are the assessments good measures of the learning goals for the course? • Are the assessments aligned with the course learning activities? • Are the performance expectations for course assessments clearly communicated to students? • Do exams ask for more than comprehension and recall (e.g., application, analysis, evaluation, creative work)? • Is the level of performance expected on assessments appropriate for this course (not too easy; not to difficult)? • Are grading criteria for exams, projects, performances, and other assignments clearly specified and followed?

9 RATING FORM FOR REVIEWING CLASSROOM INSTRUCTION (Adapted from Chism, 1999)

Course: ______Teacher: ______Reviewer: ______

While observing the class, focus your attention on the following elements of effective teaching:

A. Organization Please answer the following questions:

Needs Satisfactory Exemplary To what extent… Improvement 1. Is the teacher prepared for class (e.g., starts class promptly, has

materials ready)? ______2. Does the teacher use class time effectively/efficiently? ______3. Are learning activities well organized? ______4. Are the learning activities consistent with the course

learning goals? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements)

B. Instructional Strategies

10 Needs Satisfactory Exemplary To what extent… Improvement 1. Are teaching strategies consistent with the course learning goals? ______2. Does the teacher employ a variety of teaching methods? ______3. Is pacing appropriate? ______4. Does the teacher provide clear directions for learning activities? ______5. Does the teacher effectively integrate in-class activities and out-of-

class activities (e.g. readings, lab assignments)? ______Rate the following if appropriate for this class session. 6. Does the teacher facilitate effective class discussion? ______7. Are students given an opportunity to learn from each other? ______8. Are the board work, overheads, slides, etc. organized and helpful? ______9. Does the teacher facilitate effective group work? ______10. Is multi-media used effectively to promote learning? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements)

C. Presentation Skills

Needs Satisfactory Exemplary To what extent… Improvement 1. Is the teacher an effective presenter/facilitator? ______

11 Does the teacher appear enthusiastic (e.g., about the course, 2. the particular subject, and the students)? ______3. Does the teacher help make the subject relevant/interesting? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements)

D. Content Knowledge

Needs Satisfactory Exemplary To what extent does the teacher… Improvement Appear knowledgeable (e.g., confident about explanations and 1. answering questions)? ______Focus on the significant content of the field (e.g., uses 2. appropriate examples and illustrations, stays on topic)? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements)

E. Rapport with Students

Needs Satisfactory Exemplary To what extent does the teacher… Improvement 1. Encourages student participation? ______2. Model good listening skills? ______3. Utilize good classroom management skills? ______

12 Demonstrate personal interest in students (e.g., call them by 4. name, respond respectfully, address questions and concerns)? ______5. Respond to students’ needs and learning differences? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements)

F. Clarity

Needs Satisfactory Exemplary To what extent does the teacher… Improvement 1. Provide clear explanations and examples? ______2. Answer questions clearly and fully? ______3. Emphasize the main points of the topic or lecture? ______4. Relate subject matter to practical applications or relevant situations? ______

Comments (Summative: rationale/evidence for ratings above; Formative: comments and suggestions for improvements)

13 SAMPLE NARRATIVE OUTLINE FOR CLASSROOM OBSERVATIONS (Adapted from Chism, 1999)

Course: ______Teacher: ______Reviewer: ______

While observing the class, focus your attention on the following elements of effective teaching. (Note: not all aspects of a course will be observable on any given day.) The narrative summarizing the classroom observation should address the following areas:

A. Teacher Organization • Is the teacher prepared for class? (e.g., starts class promptly, has materials ready)? • Does the teacher use class time effectively/efficiently? • Are learning activities well organized? • Are the learning activities consistent with the course learning goals?

B. Instructional Strategies • Are teaching strategies consistent with the course learning goals? • Does the teacher employ a variety of teaching methods? • Is pacing appropriate? • Does the teacher provide clear directions for learning activities? • Does the teacher effectively integrate in-class activities and out-of-class activities (e.g., readings, lab assignments)?

Address the following if appropriate for this class session. • Does the teacher facilitate effective class discussion? • Are students given an opportunity to learn from each other? • Are the board work, overheads, slides, etc. organized and helpful? • Does the teacher facilitate effective group work? • Is multi-media used effectively to promote learning?

C. Presentation Skills • Is the teacher an effective presenter/facilitator? • Does the teacher appear enthusiastic (about the course, the particular subject, and the students)? • Does the teacher help make the subject relevant/interesting?

D. Content Knowledge Does the teacher: • Appear knowledgeable (e.g., confident in giving explanations and answering questions)? • Focus on the significant content of the field (e.g., uses appropriate examples and illustrations, stays on topic)?

E. Rapport with Students Does the teacher: • Encourage student participation? • Model good listening skills? • Utilize good classroom management skills? • Demonstrate personal interest in students (e.g., call them by name, respond respectfully, address

14 questions and concerns)? • Respond to students’ needs and learning differences?

F. Clarity Does the teacher: • Provide clear explanations and examples? • Answer questions clearly and fully? • Emphasize the main points of the topic or lecture? • Relate subject matter to practical applications or relevant situations?

15 APPENDIX C: OUTLINE FOR SUMMATIVE PEER REVIEW-OF-TEACHING REPORT

Suggestions:

 Use the section headings of the review forms as the outline for the peer review report. (The report outline below corresponds to the section headings in Appendix 2.)  Follow the expectations agreed upon in the department. Avoid interjecting personal “pet theories” of teaching or using one’s personal teaching practices as the department standard.  Focus on the effectiveness of the course design and classroom teaching in promoting student learning.  Be familiar with the statement of university expectations in Section 3.3.2C of the University Rank and Status Policy.  Support evaluative statements with specific evidence, rationale, and examples.  Include both positive and negative comments.  It is very helpful to use some type of peer review form. The department chair can determine whether to attach these forms or report numeric averages from the forms.

I. Introduction (Peer Review Process) • Who was involved? • When did the review take place? • What was evaluated?

• Course materials reviewed • Classes observed • What agreed-upon department evaluation criteria/standards were used? II. Review of Course Design • Course Content • Teaching Materials • Learning Goals • Learning Activities • Learning Assessments III. Review of Classroom Instruction • Organization • Instructional Strategies • Presentation Skills

• Content Knowledge • Rapport with Students • Clarity IV. Conclusion (Overall Assessment) • Summarize positive and negative assessments • Express overall professional judgment • How much are students learning from this teacher? • How effective is this teacher in promoting student learning?

16 • What is the likelihood that this teacher will continue to improve? • Possibly comment on patterns or trends observed in peer review results

17