Saint Mary's College of California

Revised 8-18-10

Guidelines for Department and Program Reviews

The purpose of the program review process is to provide an opportunity for departments and programs to evaluate the quality and educational effectiveness of their academic programs. In addition, the program review process is intended to promote and facilitate ongoing assessment and improvement. Changes in personnel, in pedagogical styles and resources, and in the understanding of our educational goals, suggest that we should periodically conduct such a review in order to maintain a clear vision of our goals and develop the most effective means to attain them. These reviews also afford a formal opportunity to identify needs and argue for institutional support (e.g., new tenure track faculty, lab equipment, library resources). Further, because these reviews are intended as self- evaluations to help us maintain the highest standards of education, full department participation is expected.

The guidelines below present the list of all the areas that should be addressed. The list is intended to be exhaustive; while some items may not be applicable to every program, whenever a question is not addressed, the program should explain why. To assist the Program Review Committee in its efforts to read the review carefully and consistently, program by program, please follow the exact order of topics listed below. The PRC considers the use of headings for each section helpful.

1 I. Introduction and Mission

A. What is the mission of your program?

B. How is your program consistent with the mission of the College?

C. Include a one to two paragraph history of your program.

D. Describe in one paragraph the process your program used to prepare your review.

E. Summarize and analyze your progress toward meeting the goals specified in your last program review.

F. What is your action plan for your program in the area of mission?

II. Curriculum

A. How does the program compare with its peers in the discipline in terms of curriculum (emphases, approaches, perspectives)?

B. How does the program compare with its peers in quantitative terms (e.g., number of majors, course offerings, faculty size, facilities)?

C. Is your curriculum changing in substantive ways? If not, why not? If yes, how and why?

D. What are your program’s learning goals? Have they changed since your last review? Explain

E. Explain how you evaluate the degree to which your students achieve the educational goals of your program (i.e., learning outcomes). Provide evidence.

F. Have your measures of assessment changed since your last review? Explain.

G. For undergraduate programs, briefly discuss how your curriculum meets the general education needs of non-majors.

H. Are there professional and accreditation standards in your field? Is your program accredited? Why? /Why not?

I. What is your response to recommendations made by your subject librarian in the “Review of library resources and information literacy” for your program?

2 J. Analyze the strengths and weaknesses of your curriculum in the light of the information above. What are your conclusions?

K. What is your action plan for your program’s curriculum?

Please provide the following as supporting documents in the Appendix E:

i. Sample syllabi with learning objectives. (Please make sure to include a statement on services for students with disabilities in the syllabi) ii. List of courses, majors, minors, specializations. iii. Enrollments over the last five years.

III. Student Outreach, Advising and Services

A. What additional services do you provide your students beyond curriculum? (E.g., advising, students clubs, internships, alumni organizations, mentoring, networking)?

B. How important are these services for student success and program viability?

C. What is your plan of action for student outreach, advising and services?

IV. Faculty

A. What are the faculty changes that have occurred in the last five years and how have they affected your program?

B. Do you have the faculty expertise to cover the programmatic and curricular needs?

C. What do you anticipate as your major faculty needs in the next five years?

D.Briefly discuss any challenges your faculty are facing since the last review, and if applicable, solutions you have already implemented?

E. What is your program’s approach to faculty mentoring, development and evaluation?

F. How is your program addressing issue of diversity? To what extent has it changed the way it operates due to faculty diversity? Explain.

G. Analyze your faculty strengths and weaknesses in the areas of teaching, scholarship and service. Please tell the committee about any special awards or recognition your faculty has received.

H. What is your plan of action with respect to the issues you raised in addressing A through

3 G above? Please include any short and long range plans for faculty replacement because of retirements, reduced services, administrative assignments, and leave.

Please provide documentation of the following in Appendix F: i. Number of faculty – tenure track, adjuncts, lecturers ii. Ratio of tenure track to non-tenure track iii. Ethnic diversity of faculty iv. Gender distribution of faculty v. Academic qualifications of faculty vi. Faculty participation in Seminar and Jan Term vii. Faculty curriculum vitae

V. Program Infrastructure

A. Analyze your program’s strengths and weaknesses with respect to its infrastructure. Issues that you may wish to cover in this section might include: student recruitment services, program budget, office space, administrative assistance, technical support, instructional materials and equipment, library collection and other learning resources, and information technology hardware, software and services.

B. Does your program share any interdependencies with other programs at the College of which the College needs to be aware. For example, the program may be a prerequisite for other majors, or graduate programs.

. C. What is your plan of action regarding the issues identified above?

VI. External Review Process

A. The program review must include an assessment by peers in the discipline external to Saint Mary’s College of California. The Department Chair/Program Director, the Chair of the Program Review Committee, the Dean of the School, and the Provost/Vice Provost will agree, in each case, on the form and procedures involved in the external review. It is common practice to provide assurance that there is no conflict of interest involved in the choice of professors who will conduct the external review; therefore, the Department Chair will initiate the process by providing the Chair of PRC, the Dean of the School, and the Provost/Vice Provost with a list of three schools that are being used in the review as appropriate comparison schools and the names of any professors at those schools who might be particularly appropriate to assist in the external review process. The Dean of the School will contact the potential external reviewer(s) no later than May 15 of the academic year prior to the review. One or more reviewer, as appropriate, will be selected by consensus of the four administrators named above. A written report will be submitted by the external reviewer(s) on or before December 15 to the Department Chair/Program Director. This report should be forwarded to the Dean of the School and should be appended to the Program Review.

B. The PRC encourages the external reviewer to meet with students without faculty presence.

VII. Summary Plan of Action

Please extract the action items and recommendations from the body of the report and list them here in order of priority with notes regarding the resources needed, the likely consequences to the program of not implementing each action item, barriers to implementation and the proposed timeline for completion.

VIII. Appendices

A. Copy of College Catalog entry

B. Brochures and/or advertising materials

C. Review of Library Resources and Information Literacy This is a service provided by the Library upon request.

D. External Review

E. Supporting materials for Section II. Curriculum

F. Supporting materials for Section IV. Faculty

G. Other supporting materials deemed pertinent.

5 Program Review Process Guidelines

1. The Chair of the Program Review Committee (PRC) will distribute a list by December 15 to all departments and programs (hereafter, “programs”) to be reviewed during the following academic year. 2. The Dean of the School for the program will make arrangements for external review by May 15 of the academic year prior to the review. 3. Unless otherwise arranged with the Chair of the PRC, program reviews are to be submitted by Dec. 15. Delinquent reviews from previous years are due September 1. 4. The Chair of the PRC will schedule a May workshop for chairs of all programs being reviewed in the following year. 5. All faculty in the program should participate in preparing and revising the written report. The report should contain the names and signatures of all faculty who participated in the review process. 6. Programs may request and should expect timely help from the Registrar’s office and the Office of Institutional Research concerning data pertinent to the review. 7. Program chairs may request to meet with the PRC or the PRC Chair before or after the review process has been completed. 8. Programs that undergo major external review by accrediting agencies may submit that review to the PRC in lieu of the normal Saint Mary's review. They must, however, address any items in the Saint Mary's review guidelines not covered in the external review. 9. Deans are to be given a copy of the review document when it is submitted to PRC. The appropriate Dean will be notified when a scheduled review is not submitted. 10. As specified in the Faculty Handbook (1.7.3.6.3 (g)), programs that fail to submit a review as scheduled or submit an inadequate review may be censured by the Senate at the request of the PRC, and by order of the chairperson of the Senate, all board actions concerning the program may be suspended until such time as the review is submitted and accepted as satisfactory. 11. The Dean is asked to provide a written evaluation (1.7.3.6.3 (c)) of the program review before the meeting of the PRC. 12. The PRC will meet with the chair and faculty of the program in order to discuss questions that arise from their review of the program’s materials. 13. The PRC will consider the program’s materials and produce a letter containing its conclusions, which is sent to the program Chair, the appropriate Dean, and the Provost or Vice Provost (for graduate or undergraduate programs, respectively). 14. The PRC recommends that the faculty of the program meet to discuss the PRC letter. They should address, point-by-point, the issues identified by the PRC. A brief summary of the results of this meeting should be written, including any corrections the program wishes to make, as well as any additional plan of action developed during the discussion. If the program wishes to correct any major errors in fact or interpretation, it is encouraged to write a letter to the PRC in response to the review. 15. The Provost or the Vice Provost schedules a meeting with the Dean of the School housing the program, the Department/Program Chair and faculty, and the Chair of the Program Review Committee (see Faculty Handbook 1.7.3.6.3). That meeting should focus on the main points of the program review, especially the proposed plan of action, the PRC’s response letter, and any program needs and future planning that can be identified. Every effort should be made to find a consensus about the future direction of the program and then to obtain specific 6 commitment of institutional resources sufficient to support progress towards achieving the agreed upon goals. A written record of the specific steps that will be taken to coordinate planning and resources over the next five years must be part of the process. 16. The written record produced by these two meetings specified in 12 and 13 must be filed with the office of the Academic Senate no later than the end of the academic semester following the issuance of the PRC’s letter to the program. The next program review cycle must address this material directly. Failure to carry out this step may result in the program being censured by the Senate, and, by order of the chairperson of the Senate, all board actions concerning the program may be suspended until these steps are carried out satisfactorily. 17. In each of the years following the program review, the Dean for Academic Development will ask the Department Chair or Program Director to provide an updated report on the achievement of the summary plan of action and the specific goals identified in the meeting with the Dean of the School and the Vice Provost.

Calendar of Major Deadlines

Prior Year

December 15 PRC Chair notifies programs of review deadlines in the following year.

May 1 Chairs and Program Directors supply names of potential reviewers.

May 15 Deans will contact external reviewers and arrange for fall reviews.

May PRC Chair conducts program review workshops.

Review Year

September 1 All delinquent and extended reviews are due.

October 1 Programs should contact the library, Registrar, and any other College offices in order to make a timely request of the Review of Library Resources and enrollment data relevant to the report. We also recommend contacting the Cummins Institute for assistance with aligning the department’s goal with the Catholic mission of the College.

December 15 Program Review is due. Please submit nine hard copies and one digital copy to the Office of the Academic Senate. Submit one hard copy to the dean of your school.

7 Addendum 1: Information Literacy

Information Literacy Defined

Information literacy is a set of abilities requiring individuals to "recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information." 1 Information literacy also is increasingly important in the contemporary environment of rapid technological change and proliferating information resources. Because of the escalating complexity of this environment, individuals are faced with diverse, abundant information choices--in their academic studies, in the workplace, and in their personal lives. Information is available through libraries, community resources, special interest organizations, media, and the Internet--and increasingly, information comes to individuals in unfiltered formats, raising questions about its authenticity, validity, and reliability. In addition, information is available through multiple media, including graphical, aural, and textual, and these pose new challenges for individuals in evaluating and understanding it. The uncertain quality and expanding quantity of information pose large challenges for society. The sheer abundance of information will not in itself create a more informed citizenry without a complementary cluster of abilities necessary to use information effectively.

Information literacy forms the basis for lifelong learning. It is common to all disciplines, to all learning environments, and to all levels of education. It enables learners to master content and extend their investigations, become more self-directed, and assume greater control over their own learning. An information literate individual is able to:

- Determine the extent of information needed - Access the needed information effectively and efficiently - Evaluate information and its sources critically - Incorporate selected information into one’s knowledge base - Use information effectively to accomplish a specific purpose - Understand the economic, legal, and social issues surrounding the use of information, and access and use information ethically and legally

Excerpt from: Information Literacy Competency Standards for Higher Education. (2002) Association of College and Research Libraries, American Library Association. http://www.ala.org/ala/acrl/acrlstandards/informationliteracycompetency.htm#ildef

8 Addendum 2: Learning Goals and Learning Outcomes

A) The Relationship Between Goals and Learning Outcomes

After a program’s mission statement has been established, the next step for faculty is to design goals and the intended outcomes for those goals. Faculty should strive after goals that can easily be converted into learning outcomes. It might be helpful to note that while goals tend to focus on delivery of information (Students will become familiar with the cultures of ancient Greece and Rome), the converted outcomes focus on effect (Students will compare and contrast the political, economic, and religious systems of Greece with that Rome). In order to arrive at outcomes, faculty might consider the following questions as a guide for the learning outcomes discussion:

1) What do we want students in our major to know? 2) What do we want our students to be able to do? 3) What values or attitudes (dispositions) do we want to instill in our students?

The statement of learning outcomes at the program level clarifies for all stakeholders the knowledge, skills, and abilities a student must possess to successfully complete a course or program and thus earn a degree from the College. Learning outcomes have three distinguishing characteristics: the learners’ specified action must be observable, measurable, and performed by the learners. Here are examples of goals followed by their respective outcomes:

• Students will be familiar with the syllogism. • Students will have an appreciation of cultural diversity in the classroom. • Students will be familiar with the different kinds of writing genres.

These goals can easily be converted into outcomes by inserting action verbs and in some instances making clear the predicate.

• Students will conduct logical arguments through the use of a syllogism. • Students will summarize in writing their attitudes about cultural diversity in reflective essays. • Students will submit by graduation the following kinds of writing: essay, creative non-fiction, summaries, paraphrase, and reports.

Since the learner’s performance ought to be observable and measurable, the verb chosen for each outcome statement should result in overt behavior that can then be observed and measured. Sample action verbs are: compile, create, plan, revise, analyze, design, select, utilize, apply, demonstrate, prepare, use, compute, discuss, explain, predict, assess, compare, rate, critique. Conversely, some verbs that are unclear subject the learning statement to different interpretations in terms of what action is being specified; such verbs, which obscure the intended behavior, cannot be observed or measured. These types of verbs should be avoided when constructing learning outcomes: know, become aware of, appreciate, learn, understand, become familiar with.

B) How to Establish Goals and Learning Outcomes

9 The most practical way to start the conversation about goals and learning outcomes is to weigh the needs of the program mission statement against examples of outcomes published through the respective professional organizations or by departments with similar programs at other schools. This process should help generate a comprehensive list of ideas and suggestions for learning outcomes that can then be refined and narrowed. Once the department has generated such a list, the most challenging task is then deciding which goals and outcomes are essential at the program level. Practitioners point out that if the final list of learning outcomes initially reaches beyond five or six, the assessment effort will be onerous, even unmanageable. During discussions, it might be helpful to keep in mind that all outcomes should be measurable. Program goals articulate both measurable and non-measurable expectations for students; outcomes must be expressed in terms of demonstrable characteristics. Outcomes focus on what students in the major can demonstrate rather than on what faculty members teach. The list of learning outcomes can always be reviewed and revised as the need arises or as new developments occur in the discipline. In any event, it important to make sure that students know the program’s learning outcomes. Research has shown that students who are aware of the direction of their education are much more likely to be engaged in the learning process.

Here is a model of learning outcomes for a History department:

1. History majors will demonstrate skill in chronological thinking. 2. History majors will demonstrate historical comprehension. 3. History majors will demonstrate the skills of historical analysis and interpretation. 4. History majors will demonstrate historical research capabilities. 5. As a consequence of these skills, History majors will demonstrate a) a working knowledge of the natural and cultural environment in which humans have developed and live; and b) a sense of the diversity of the human experience influenced by geography, culture, race, ethnicity, gender, and class. 6. History majors will demonstrate an understanding of the career search in appropriate educational and professional markets

C) Use Curriculum Mapping to Confirm Outcomes

Once the preliminary list of outcomes is established, faculty should engage curriculum mapping to see where those outcomes are embedded in current courses. A curriculum map consists of a table with two axes, one listing program outcomes, the other listing courses in the major. The coverage of a learning outcome in each course is shown in the cells of the table (see Table 4.1)

Table 4.1. Curriculum Map Template

Program Learning Course 1 Course 2 Course 3 Course 4 Course 5 Course 6 Course 7 Outcomes Outcome 1 X X X X X X X Outcome 2 X X X Outcome 3 X X X X Outcome 4 Outcome 5 X X X

10 Course mapping can provide a view of how individual courses are related to the program learning outcomes and can show the curricular emphasis given to each outcome. For example, Outcome 1 in the above table appears in seven courses and is therefore given greater emphasis than Outcomes 2 and 3. A map can also identify program redundancies or gaps, such as the fact that no course currently takes ownership of Outcome 4.

Curriculum mapping can help facilitate faculty discussion about the extent to which the program currently addresses the list of learning outcomes. A different kind of map, one that indicates levels of intensity, can go even further by displaying the degree to which a course emphasizes a specific outcome. In some cases, it may even be useful to show the number of hours devoted to each outcome in each course; the level of achievement that is expected in each course can be indicated as low, medium, or high or other such descriptors. This would reveal how student achievement is expected to develop during their progress through the curriculum.

*For those who are need further convincing about the importance of outcome-based pedagogy, read the following noteworthy articles.

Articles: From Teaching to Learning - A New Paradigm for Undergraduate Education By Robert B. Barr and John Tagg. November/December 1995. Change magazine. Found at: http://critical.tamucc.edu/%7Eblalock/readings/tch2learn.htm

So, What’s a Learning Outcome Anyway? Learning Outcomes and the Learning Paradigm By Mark Battersby, Department of Philosophy, Capilano College. Found at: www.pierce.ctc.edu/Library/lstagrant/wp-content/battersby.doc

Reprint of document found on the web: Daly, P. (2005). Establishing Learning Outcomes. In Assessment of Student Learning: Assessment Handbook [Web]. Indiana University Southeast. Retrieved May 7, 2007, from http://www.ius.edu/assessment/pdf/Handbook/Chapter4.pdf

11 Addendum 3: Services for Students with Disabilities

Services and reasonable accommodations are available to students with physical or mental impairments or conditions that qualify as a disability and which impede the equal opportunity to participate with other students at Saint Mary's College of California. Verification of a disability, its nature and the appropriateness of the requested accommodation in relation to the disability either through a personal physician, a licensed learning specialist, or a licensed and credentialed mental health professional must be submitted to Saint Mary's 504 Coordinator, located at Academic Advising and Achievement Services Office in De La Salle Hall.