Student Learning Outcomes: Considering the New Standards

Student Learning Outcomes: Considering the New Standards (A Presentation at the LACCD Retreat, October 24, 2003) by Greg Gilbert

I am here today representing the Academic Senate for California Community Colleges, where I serve as Chair of the Research Committee and Liaison to the Accrediting Commission. As Research Chair, one of my charges is to work with others to create a set of resources to assist local senates as they undertake their self-studies. My preparation includes a background in assessment and accreditation. I am an English professor and past president of the Board of the Morongo Unified School District. I have overseen the development and implementation of large scale writing portfolio projects at the university and K-12 levels, and I have chaired standards on several accreditation reports. One point of pride is that within my local school district, I led the charge to keep local curriculum and assessment decision under the control of teachers. Even so, the Federal Government and the State have left very little room for local authority over anything. During today's presentation, I will begin with a rather substantive note of caution about accountability trends in our state. This will be followed by a discussion of the faculty’s role in accreditation, as well as challenges to our local authority, a review of WASC’s function, and some preliminary advice on how local senates might choose to engage the new standards. Certainly, when we discuss accreditation, we must consider it within the spectrum of a number of issues that are playing themselves out in California. I should start by emphasizing one point. The Academic Senate for California Community Colleges has nothing against assessment – per se. In fact, we like assessment. We like assessment that encourages students to reflect on their mastery of course materials. We like authentic assessment that demonstrates progress over time, assessment that is valid and reliable, assessment that respects the use of traditional measures. What we do not endorse and will not accept is a uniformed and authoritative imposition of tests upon our classrooms – a one-size-fits- all approach. Unfortunately, such a concern must serve as the foundation for any substantive consideration of the new standards. Disheartening though it may be, there is in California a multi-pronged endeavor to diminish our right to enact local decisions on curriculum and programs. Never has there existed a greater incentive for board members, administrators, and faculty to focus on their common goals in service to students. Never has there existed a greater incentive for all of us to work together as colleagues, particularly on our self-studies. As we all know, SB 6 is intent on establishing a commission that would coordinate the statewide articulation of curriculum and assessment among all segments of education. While we cherish the promise embodied in our long-standing Master Plan for Higher Education and know that it has well served our diverse students, SB 6 is problematic for our community college system. Its sponsor, Senator Alpert, has yet to provide clear justification regarding its central purpose, nor are we certain of her intentions. As we consider SB 6, along with WASC’s imposition of Student Learning Outcomes (SLOs) and the legislature’s intent to establish Accountability Frameworks as a guide for budgetary decision making, we see a pattern emerge wherein aggregated data will determine individual local choices. And within the new accreditation standards, we see a systematic removal of language that differentiates employees and replaces the governance standard with a

Gilbert/LACCD 1 of 8 standard referring to "Leadership and vested authority.” In sum, SB 6 and accountability frameworks are empowering decision-making by people who have never met our students or stepped foot onto our campuses. As for the new standards, when we couple their disregard for collegial consultation with their insistence that SLOs be used in the evaluation of faculty, we have just cause for alarm. Given our collective service to students, one must wonder why such changes are being unilaterally imposed. Consider how well our existing system has dealt with its extraordinary regional diversity. Of the 2.9 million students that we serve state-wide, 20% are recent immigrants or English-language learners. Seventy-five percent of students-of-color in California who are pursuing higher education goals are doing so within our community college system. We are an education-on-demand institution. Our mission is centered on open enrollment, affordability, and regional diversity. Traditional measures have favored inputs over outcomes, and the current managerial ideology fails to recognize that our profession has done a yeoman’s job of preparing people for certificates and degrees, for personal growth, and to sustain California’s public services and infrastructure demands. What we need to recognize is that any effort to impose standardization on our colleges and classes should be viewed as an affront. If standardization occurs, even regionally, the likely result would be a limited range of textbook selections to match a narrow set of exams for every subject area. Experts would for a rather immodest sum come to our campuses and provide guidance so that we could all improve together. Could such an Orwellian theme actually play itself out in our society? We have only to look as far as our K-12 system and “No Child Left Behind” to know that it could – right here in America. How wonderful it all sounded in the beginning – no child left behind. Everywhere, the same curriculum, aligned vertically and horizontally. One set of standards, and all of the tests and all the consultants that a well-aligned society could ever desire. Everybody on the same page. High stakes mandated assessment within the K-12 system has given rise to an industry of test designing, proctoring, assessing, and textbook and supplemental materials that tend to favor business people but offer little benefit to students. In addition, the sudden blizzard of testing statistics is providing data for politicians, grant writers, entrepreneurs who would open private schools, and for those who would form institutions to intervene with failing schools, offer conferences to raise test scores, train new administrators, and assess those teachers whose students are not making the grade. When one considers that the imposition of testing and its resultant money trail came into being without student-centered research or a substantive dialogue with educators, a financial impetus of the movement becomes apparent. Certainly, there is ongoing debate as to whether the K-12 debacle is a harbinger of things to come for community colleges or not. Remember, the slippery slope toward K-12’s loss of control over curriculum and assessment began when faculty attempted to comply with a demand to craft assessments that functioned beyond their classes, at site and district levels. Such a charge was not within the expertise of classroom faculty, for the most part, and it involved their having to negotiate district politics, issues of validity and reliability, course and program design, and all of the extra hours involved in an attempt to meet such challenges. As the situation that was foisted upon faculty began to

Gilbert/LACCD 2 of 8 unravel, the state and federal government and their consultants stood ready to provide assistance in the form of large-scale testing and a narrowed curriculum. Since Fall, 1979, the Academic Senate for California Community Colleges has passed nearly 120 resolutions dealing with accreditation. Of those resolutions, nearly one-third date from Fall 2000 and urge opposition to the unilateral imposition of the new standards and their reliance on SLOs. Though the Senate will continue its dialogue with WASC and the Accrediting Commission for Community and Junior Colleges (ACCJC), as will other institutions of higher learning, the standards have been adopted and are now the official guide for all accreditation self-studies. I should hasten to emphasize, however, that the old standards were guided by the Ed Code, by Title 5, by AB1725, and by a range of matriculation and articulation supports. Never did the Commission attempt to insert itself into classroom instruction and assessment practices. Even the most narrowly focused legislator or statistician should be able to grasp the implication here. Finally, and most chilling, we see language that says that “Faculty and others directly responsible for student progress toward achieving stated student learning outomes have, as a component of their evaluation, effectiveness in producing those learning outcomes” (III.A.1.c). Is collective bargaining to be swept aside along with academic freedom and local control? Certainly, as we develop our self-studies we must embrace the freedom that we yet retain. On Thursday, October 2, 2003, Chancellor Nussbaum, in his annual address, said, “After 28 years with the California Community Colleges, I do have to say that local control gets it right 95 percent of the time in 95 percent of the districts.” Thus while we retain our local authority, we will continue to get it right, and that is where we must seize the initiative with the new standards and their culture of evidence. Fortunately, for the state, our local senates and teaching professionals can help sort things out, provided the state is willing to work with us in a cooperative manner. As we know, assessment and accountability have always been fundamental to the teaching profession, so first and foremost, we are resolved to provide outcomes and measures that support student learning while NOT diminishing the curriculum or narrowing the definition of our student base. In other words, we may select to help the state and the ACCJC acquire the information that they desire, but not at the expense of academic freedom and our responsibility to our students.

Several Basics: First and foremost, the accreditation process will remain faculty driven. This is not just a good idea; it’s the law, at least for the present time. The dyad of student and teacher is an ancient and sacred union that links the past to the future. Subject knowledge and critical reasoning skills are not innate characteristics; rather, they, like language itself, are a legacy passed from one generation to the next. Within this legacy exits the cultural and educational components that are the bedrock of our civilization. Faculty are the subject-matter experts and, equally important, the ones who work with students every day. Because we know our subjects and our students, it is appropriate that the state has established laws that support our self-evident right to consult on all matters academic and professional. The Education Code and Title 5 provide the basis for our authority. AB 1725, the bill that decoupled community colleges from K-12, repositioned us within the state’s Master Plan for

Gilbert/LACCD 3 of 8 Higher Education. As a result, minimum qualifications were raised and probationary periods were extended. Peer review was attached to faculty evaluation and funding was established for professional development. Therefore, it is appropriate that the Academic Senate is an organization whose primary function is to make recommendations with respect to Academic and Professional matters, many of which are referred to in the 10+1 and serve as the basis for college governance policies that are established between local senates and their governing boards. Within the 10+1, Accreditation is item seven. To Consult Collegially means that the district governing board shall develop policies on academic and professional matters through either or both of the following: 1. Rely primarily upon the advice and judgment of the academic senate, OR 2. The governing board, or its designees, and the academic senate shall reach mutual agreement by written resolution, regulation, or policy of the governing board effectuating such recommendations.

1. Curriculum, including establishing prerequisites. 2. Degree and certificate requirements. 3. Grading policies. 4. Educational program development. 5. Standards or policies regarding student preparation and success. 6. College governance structures, as related to faculty roles. 7. Faculty roles and involvement in accreditation processes. 8. Policies for faculty professional development activities. 9. Processes for program review. 10. Processes for institutional planning and budget development. 11. Other academic and professional matters as mutually agreed upon.

In either instance, local senate expertise is paramount.

According to the State Academic Senate’s 1996 paper, “Faculty Role in Accreditation,” Strong faculty leadership in the development of the self-study is of utmost importance to its integrity. Title 5 (section 53200-204) clearly indicates that local senate participation in the accreditation process is part of collegial consultation, the cornerstone of shared governance. The local senate should be involved in the development of the self-study plan, including the committee structure. Also the local senate appoints faculty to the self-study committees. [. . .] The subcommittees, one for each standard, are chaired by a faculty member or by a faculty member and either an administrator or classified staff person sharing responsibilities. Faculty serving as chairs or co-chairs should be appointed by the local senate (See Title 5, section 53206) in consultation with the college president. Subcommittees are responsible for developing drafts of the reports for each standard. Each subcommittee should have input from all of the institution's constituencies. More people involved in this work means less each person will need to do. Also, it is very important that each of the school's employees feels that he or she shares responsibility for the development of the self-study.

Gilbert/LACCD 4 of 8 Now, briefly, what is accreditation? (Excerpted from the WASC website): The Western Association of Schools and Colleges (WASC) is one of the six regional accrediting associations covering the United States whose purpose is continual improvement of education and cooperation among educational institutions and agencies. WASC was formed on July 1, 1962 to evaluate and accredit schools, colleges, and universities in California, Hawaii, American Samoa, Guam, the Commonwealth of the Northern Marianas, the Republic of the Marshall Islands, the Federated States of Micronesia, and the Republic of Palau. WASC functions through a board of directors and three accrediting commissions: the Accrediting Commission for Senior Colleges and Universities, the Accrediting Commission for Community and Junior Colleges, and the Accrediting Commission for Schools. The Accrediting Commission for Community and Junior Colleges (ACCJC) accredits institutions in the WASC service area offering the undergraduate curriculum at less than the baccalaureate level. The ACCJC accredits only institutions that offer an associate’s degree. The Commission evaluates the institution’s performance against the Standards of Accreditation and in the context of the institution’s own mission and accredits institutions, not individual programs.

What Shall We Do? Now, we have spoken of our authority under the law and the role of the ACCJC. We have also said that we will NOT, in effect, comply with a mindless clerical exercise to chase down numbers. What, then, is it that we will do? First, we must recognize that the new standards offer local senates the opportunity to strengthen their role in college governance with regards to matters academic and professional, and this is by exerting authority over the process and demonstrating the effectiveness that can be found in traditional measures.

A Project Plan There is within the standards a kernel of hope. The four standards represent a fairly typical approach to project planning in that they require a mission that is linked to resources, production, and authority. The project, in this instance, involves SLOs, and while college administrators manage resources, the role of the faculty is essential to the completion of an institution's successful accreditation report. As I have said, administration and faculty must work together, cooperatively and constructively, as they do at many colleges every day.

Standard I Standard II Mission Student Effectivenes Learning s Programs and Services

Standard III Standard IV Resources Leadership & Governance

Gilbert/LACCD 5 of 8 In effect, a mission may be broken into component parts. What is it promising? Who does it serve? How effectively is it implemented? What feedback mechanisms are in place to help it deliver on its promises? How well does the mission match with the institution’s master plan? How effectively do local governance roles support institutional effectiveness? When faculty hold the institution accountable to its mission, they operate from a basis of genuine authority. Thus, at the institutional level, the faculty can connect the mission/vision to a matrix or plan that delineates the community being served (certificate, degree, transfer, life-long learners, DSPS, EOPS, etc.) and matches that community to specific services, such as counseling, tutoring, and programs. In addition, faculty may consider quality control issues as they relate to the promise of the mission (rigor, retention, compression of courses, platforms for courses, etc), and all of this can relate to the content of course outlines of record, and program funding and discontinuance. Within the college’s mission and board policies are ideals that point toward student success – and this is where local senates can use the self-study to best advantage. The more dicey accreditation areas exist at the program and course level, and this is where faculty must make certain that their reliance on traditional measures is well honed. Some colleges, like San Diego Miramar and its 21st Century Program, have established core competency skills and measures that can be adapted to other colleges, but the fact remains that even in assimilating an already developed program, a great deal of work is entailed. In order to comply with the demand for SLOs, faculty will be expected to redesign their course outlines of record so that they contain measurable outcomes. Faculty will be required to include SLOs in the weave of their program reviews and to construct reliable evidence that demonstrate the validity of their assessments. At the classroom level, assessment has always been somewhat less exacting than when we move toward institutional measures. Classroom assessments are often colored by our individual pedagogies and the insights we have regarding our students. Statisticians may believe that our individualistic approaches are a weakness, but we know otherwise, and that goes to the very heart of our need to retain authority over the accreditation process. Our students and our traditional measures are what real teaching is all about and not the collecting and crunching of soulless numbers. If we are to maintain our adherence to traditional measures, we must articulate them well. A grade will need to be understood concretely by someone external to our classrooms. But we can do that. At the classroom level, at the department level, and between departments (and colleagues), there exists a wonderful opportunity to work together, to go cross-curricular, to establish measures that are valid and authentic. We can create rubrics that retain all of our important and traditional values, and we can use them to measure progress over time, such as when we compare a recent writing sample to an earlier one, or a set of competencies in public speaking, or reading for critical analysis, or by using portfolio work that contains a reflective analysis by the student – all ideas that can function beautifully within the paradigm of traditional measures. As one of my colleagues said, “All that is required is the establishment of a healthy atmosphere of doubt.” Finally, SLO data can be collected and aggregated without admitting any reference to the specific class and its instructor. As for those colleges that have already taken the first steps within the milieu of the new standards, information is surfacing. Among those colleges is the College of the Siskiyous.

Gilbert/LACCD 6 of 8 Recently, Dave Clarke, the Academic Senate President, provided the Commission with a close reading and marginalia for the guidelines on Standard IV. He pointed out, quite rightly, that some of the commission's supportive language has the appearance of ambiguity and is not easily understood. The faculty at the college had a long and fruitful discussion with Barbara Beno, the ACCJC's Executive Director, and Darlene Pacheco, the Associate Director, and the bottom line, according to Dave, is that the report should entail a broad and emphatically ordered narrative that references, fairly thoroughly, the guidelines.

The advice that Dave asked me to pass along is as follows: 1) Standards chairs should be provided with release. This is a big job. 2) The self-study is not about isolated segments, bullets, and questions. Everything is interconnected and standards cannot be addressed in isolation. 3) Treat each committee as a focus group and bring back information to a central group for consideration and coordination. 4) The report is to be a narrative that uses the guidebook's questions as a heuristic. Select one writer and have everyone else bullet their information for that person. 5) While the process should be faculty driven, administration must be genuinely involved and not peripheral to the process. 6) Chairs must be supported by faculty and administrative authority when they request information and establish deadlines. 7) Because this report depends on narratives, avoid empty surveys that ask people to select points along a simple scale of values.

Having said this, other colleges have reported being told that what is wanted are models of measurements, and some Accreditation Liaison Officers (ALOs) are expressing frustration and viewing the standards as confusing and impossible. For now, we are all learning together. Hopefully, every college will send representatives to the ASCCC Plenary at the Pasadena Hilton on October 30-November 1. On Thursday, Chancellor Nussbaum will speak, and there will be two breakout sessions on the new standards, one a presentation of San Diego Miramar’s 21st Century Program, and another breakout which will involve a round table discussion with those colleges who have been piloting the new standards; the second session is entitled: “The New Accreditation Standards: Been There; Done That.” Simply stated, the demand for accountability is at odds with our regional differences, with the short duration of our programs and classes, and with open enrollment. As any statistician knows, selectivity is the best predictor of success. Look at what it takes to get into an ivy-league university, and we begin to appreciate that a transient population, open enrollment, and a wide- range of diverse backgrounds and goals cannot lead to consistent and valid predictors. The attempt to "validate" course examinations requires an enormous investment of time and institutional resources, but if faculty work together and assessments are authentic, cross- curricular, and normed, the result can be better coordinated instruction and a more fully integrated curriculum without hiring a coven of researchers at every institution. Unfortunately, some colleges in California are hiring researchers instead of filling sorely needed faculty positions, but by taking the initiative on self-studies, we may forestall such an event at our own colleges.

Gilbert/LACCD 7 of 8 The view of the ASCCC, as noted in the “Guidelines for the Field” (www.academicsenate.cc.ca.us), “we are not victims,” and, in practical terms, “we have to take control of the process.” The Senate’s paper, “The Faculty Role in Planning and Budgeting” (Fall 2001) says, “Planning, coupled with critical assessment of successes and failures, is a means of taking conscious control of the process of serving students, and enables the emergence and elaboration of best practices.” While we should welcome the challenge of cyclical evaluation, planning, implementation, and re-evaluation, we must, as Mark Snowhite (ASCCC’s Secretary) writes in the April “Rostrum,” be concerned with “[. . .] who determines the [. . .]SLOs and who decides how and when we use them.”

Hopefully, accreditation will transcend a clerical exercise in compliance and, instead, embody our deepest values. It will do so if it:  facilitates reflective and authentic assessments;  provides information that can be used to strengthen programs and instruction;  results in consistent and measurable definitions of rigor and student success;  contributes to course designs that value quality ahead of an institutional hunger for revenue;  maintains the roles of faculty and students at the center of institutional planning;  steers clear of any standardization of instruction and assessment; and  respects the need for barriers between collected information and faculty evaluations.

In conclusion, I will leave you with the words of Daniel J. Boorstin who said, “Knowledge is not simply another commodity. To the contrary, knowledge is never used up. It increases by diffusion and grows by dispersion.”

Gilbert/LACCD 8 of 8