Youngstown State University

Youngstown State University BEEGHLY COLLEGE OF EDUCATION Evidence-Based Program Improvement Plan Form

STAGE 1: Program Issue and Analysis

This form is used to record all program evaluation efforts. This form along with any relevant attachments is to be sent to the department chair who will forward it to the Performance Assessment and Program Evaluation Committee (PAPEC) for review and for recording it as documentation of our commitment to using evidence and data to monitor and improve our programs.

Name of Program: ____AYA Initial Licensure______Today’s Date: __09/03/09______

This Program is: Undergraduate __X___ Advanced: _____

Submitter: ___Howard Pullman______

This Evaluation Review is: in Progress ______Completed ______Ongoing __X___

Year and Semester: _Spring 2009______

Date of Initial Evaluation Review if Conducted: ___Not Identifiable______

Specific Issue Addressed:

The Department of Teacher Education dropped the required course in Tests and Measurement effective Fall 2000 when YSU changed from quarters to semesters and the Ohio Department of Education changed its teacher certification procedures to licensure procedures with performance assessment of entry year teachers using Praxis III. At that time it was felt that educational assessment issues and procedures could be integrated into all education courses. This did not prove to be a satisfactory solution. This PIP documents the reintroduction of a course in Educational Assessment into the AYA programs and its grouping with Principles of Teaching Adolescents; Reading Application in Content Areas, Secondary Years; and Special Methods – Content Area into the field-intensive Secondary Cluster semester.

I. Evidence and Data source(s):

Data relevant to the decision to reintroduce a course in assessment comes from ongoing program evaluations completed by student teachers, cooperating teachers, and university supervisors; student teaching performance evaluations completed by cooperating teachers and university supervisors; beginning in Spring 2005, ratings of the Teacher Work Samples; and to some extent the results of the Praxis II PLT for grades 7-12. Discussions in department and AYA program meetings also provided evidence of the need for reintroducing a formal course in assessment.

II. Methods and Procedures used to Analyze the Issue:

Prior to the formation of the AYA Program Committee data/evidence from the above sources were made available through the Office of Student Field Experiences, TaskStream, and ETS reports of Praxis II scores. Data were summarized and reported/discussed at Department of Teacher Education faculty meetings. With the formation of the AYA Program Committee, data were analyzed by this smaller group and more lengthy discussions of curricular solutions were engaged in by those faculty who would ultimately be most directly involved in the implementation of any such solutions. The final curriculum proposal was taken from the group to the department as a whole for initial approval. III. Findings and Significance:

The following excerpts are taken from the Department of Teacher Education annual program assessment reports to the YSU Assessment Council. These excerpts document the findings of the AYA Program Committee up to the initial implementation of the cluster in Fall 2008.

Program evaluation completed by student teachers indicates that they feel least prepared in assessment and classroom management. Ratings of lack of preparedness increase with the age of students the teacher candidates are preparing to teach. These ratings are mirrored by program evaluations completed by school and campus supervisors.

As a result of the data gathered from the Teacher Work Sample evaluations, we have begun discussion about reinstating a course dealing with assessment into some or all of the initial licensure programs. This has come about in the Standard 1 and Standard 2 Committee meetings as well as informally among faculty involved in evaluating the Teacher Work Samples. (Assessment Council Report, 2005)

This reflects a noticeable change from last year’s results when assessment was also identified as an area of major concern to student teachers. This change may be accounted for by the fact that student teaching seminar leaders have made an increased effort to address assessment issues in the context of the Teacher Work Sample.

Finally, the Teacher Work Sample, a culminating project, was implemented two years ago as an accompaniment to student teaching. The Teacher Work Sample is scored both analytically and holistically. Preliminary results indicated that our students were weakest in designing assessment procedures and interpreting the results of assessment. This is consistent with evaluations of the initial licensure program by student teachers, cooperating teachers, and university supervisors.

Students’ weakness in designing assessment procedures and interpreting the results of assessment is being addressed by seminar leaders in the student teaching seminar. The result of this effort is apparent in the evaluations of the Teacher Work Samples. However, many faculty believe that it is only with earlier and more extensive efforts that the underlying need will be met. The effects of addressing this weakness through the proposed reinstatement of a required assessment course will not be known for at least a year after it is implemented. (Assessment Council Report, 2006)

This reflects a noticeable change from previous results when assessment was also identified as an area of major concern to student teachers. This change may be accounted for by the fact that student teaching seminar leaders have made an increased effort to address assessment issues in the context of the Teacher Work Sample.

Finally, the Teacher Work Sample, a culminating project, was implemented three years ago as an accompaniment to student teaching. The Teacher Work Sample is scored both analytically and holistically. Preliminary results indicated that our students were weakest in designing assessment procedures and interpreting the results of assessment. This is consistent with earlier evaluations of the initial licensure program by student teachers, cooperating teachers, and university supervisors.

As a result of the data gathered from the Teacher Work Sample evaluations; program evaluations by student teachers, cooperating teachers, and university supervisors; NCATE Standard 1 and Standard 2 Committee meetings; and informal discussions among faculty, the department approved the revision and reinstatement of a 3 semester hour assessment course – FOUND 3710 Educational Assessment – into programs which prepare teachers of students in secondary schools. This course will be integrated into a new field-based semester for secondary and career/technical licensure candidates. This “Secondary Cluster” will begin in Fall 2008 and will also include Principles of Teaching Adolescents (SEDUC 3706), special content methods (SEDUC 4800X), reading in secondary content areas (TERG 3711). The assessment course was eliminated from all programs during the transition from quarters to semesters. (Assessment Council Report, 2007) PLT data do not show that students perform noticeably worse of the assessment related subtests, although that may have been expected from previous discussions.

Although the mean rating for TWS elements in the planning for assessment process remains the lowest of the seven processes, it is no longer noticeably lower than the means of the other six processes. It is interesting to note that the highest mean rating was for a related process, analysis of student learning, which requires teacher candidates to analyze results of the assessment plan implemented during and following instruction.

Data from the TWS, along with data from other sources, have led us to reaffirm the need for instruction in assessment as a formal component of our initial licensure program. This has resulted in curriculum actions adding FOUN Educational Assessment to one of the field intensive semesters (Adolescent/Young Adult)

Previous surveys reported that assessment was by far the lowest rated category by all three groups. That is no longer the case. The following table summarizes percents of student teachers (ST), university supervisors (US), and cooperating teachers (CT) who responded Very Satisfied or Satisfied with the student teachers’ performance/preparation in the areas of planning, classroom management, knowledge of content, instructional differentiation, and assessment.

Planning Management Content Differentiation Assessment ST US CT ST US CT ST US CT ST US CT ST US CT 52 21 27 44 26 35 42 16 26 47 30 30 46 28 32 Satisfied 38 74 65 37 68 53 53 82 67 43 67 61 49 67 59 Very Satisfied 91 95 93 81 94 88 95 98 93 91 97 91 96 95 91 Sum

(Assessment Council Report, 2008) Youngstown State University BEEGHLY COLLEGE OF EDUCATION Evidence-Based Program Improvement Plan Form

STAGE 2: Program Action(s) and Oversight Plan

Name of Program: ____AYA Initial Licensure______Today’s Date: __09/03/09______

This Program is: Undergraduate __X___ Advanced: _____

Submitter: ___Howard Pullman______

Year and Semester of Initial Evaluation Review: _____ Not Identifiable ______

Specific Issue Addressed:

The Department of Teacher Education dropped the required course in Tests and Measurement effective Fall 2000 when YSU changed from quarters to semesters. At that time it was felt that educational assessment issues and procedures could be integrated into all education courses. This did not prove to be a satisfactory solution.

IV. Program-Specific Recommendations/Actions based upon Stage 1 (Use attachments if needed):

The following excerpt is taken from the program proposal for the Secondary Teacher Education Cluster which was approved by the required program, department, college, and university program/curriculum committees during the 2006-2007 academic year. As planned, the cluster was implemented in Fall 2008.

The intensive semester clusters together four courses in those teacher education programs which include SEDUC 3706 Principles of Teaching Adolescents. Three of these courses are currently required; this proposal represents a reordering of these courses and the inclusion of a course in educational assessment. The four courses represent general principles and methods of teaching adolescents, specific content (e.g. mathematics) methods of teaching adolescents, reading in the content area for teachers of adolescents, and educational assessment. The gains to be realized by clustering these courses in the semester prior to student teaching include the integration of field experiences into all four courses; coordination of instruction and assignments between general and content specific pedagogy, e.g. reading and mathematics, assessment and social studies; and team teaching opportunities between instructors in general and content specific pedagogy courses. The fourth course, educational assessment, is a reinstatement of a previously required course with a concomitant reduction in the hours of one of the existing courses, SEDUC 3706. The assessment course was dropped from the program when YSU converted from quarters to semesters. The decision to reinstate an assessment course comes through: a. student teacher evaluations of the teacher preparation programs, b. cooperating teacher and university supervisor evaluations of the preparation of student teachers participating in these programs, c. faculty evaluations (from the BCOE Assessment System) of the student teachers’ ability to plan and carry out assessment of P-12 student performance, d. federal No Child Left Behind legislation mandating extensive testing of P-12 students, and e. mandated state Value Added Assessment procedures that track federal mandates. The net result of this organizational modification is the addition of one semester hour to programs in the following licensure areas: Adolescent/Young Adult; Career/Technical; Multi-Age - Health, Foreign Languages. Licensure candidates in some of these areas will be able to select elective or optional courses in other parts of their programs so that there is no net change in the total required number of hours; candidates in other programs may not be able to do so. In either case there should be no additional cost to licensure candidates associated with this change. V. Does this Program Recommendation/Action require a Formal Review by the Undergraduate or Graduate Curriculum Program/Curriculum Committee? Yes __X___ No _____

The addition of the assessment course and the concomitant reorganization of courses into the Secondary Teacher Education Cluster was approved by the required program, department, college, and university program/curriculum committees during the 2006-2007 academic year.

VI. Oversight and Monitoring Plans for the Program-Specific Actions:

The Teacher Education AYA Program Committee, including all faculty teaching in the cluster, reviews program data each semester. Sources of these data include the Teacher Work Sample; program evaluations by student teachers, cooperating teachers, and university supervisors; student teaching observations by cooperating teachers and university supervisors; and the Praxis II 7-12 PLT.