Assessors Anonymous Handbook

College of Education University of Hawai‘i at Mānoa

Assessors Anonymous Handbook for 2009 Focused Visit

A Local Guide to Continuous Program Improvement (and Accreditation Reports)

QuickTime™ and a decompressor are needed to see this picture.

QuickTime™ and a QuickTime™ and a decompressor decompressor are needed to see this picture. are needed to see this picture.

QuickTime™ and a decompressor are needed to see this picture.

Oh, sorry, did you say rubric?

Fall 2008 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Table of Contents: The AAAAA List

This Guide is designed to assist COE departments and programs in putting together assessment plans and accreditation reports. We hope other units find it useful, as well.

FIVE STEP PLAN Adapted from a presentation by Douglas Walcerz and Eric Gibbs, True Outcomes, May 2008, UHM

1. ACCOUNT (page 3) Anchor your program (provide your context) in COE. (What is it all about, Alfie?)

2. ALIGN (5) Agree on what you are trying to accomplish. (Yes, this does mean your faculty.)

3. ASSESS (7) Accumulate evidence. (Use technology to avoid perishing in piles of paper.)

4. ANALYZE (10) Aggregate; disaggregate. (And aggravate just about everyone in the process.)

5. ACT (11) Apply what you find to help students learn better. (Worth it.)

Appendix A Template Workbook (12) Appendix B COE Program Examples (16)

Oh, yes . . . NCATE Focused Visit in Fall 2009

2 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Step 1. ACCOUNT Anchor your program (provide your context) in COE

Describing your program comes first, including the usual suspects—who, what, when, where, why, how. Set the context for your program, students, faculty, and program standards or outcomes as an essential part of the College of Education. Remember to link what your program does to the COE vision, mission, and conceptual framework, shown here.

College of Education Motto: Preparing educators to contribute to a just and democratic society Vision: The COE consists of educators who provide innovative and cutting-edge research and teaching in an effort to further the field of education and prepare educators to contribute to a just and democratic society. Mission: The mission of the College is to work as a diverse and democratic community to: • prepare new educators and provide ongoing professional development in education (teaching), • increase the knowledge base in education and related fields through the production and application of research related to teaching, learning, and assessment (research), and • serve as partners and leaders for excellence in education (service). Philosophy, Purposes, and Goal: The College is committed to the fundamental goal of employing and preparing education professionals who are knowledgeable, effective, and caring. These individuals demonstrate these core values through their knowledge, skills, and dispositions.

COE programs that submit program reports (e.g., Special Education, Secondary Mathematics, Educational Technology) to the National Council for Accreditation of Teacher Education (NCATE) Specialty Professional Associations (SPAs) for review must provide the following information in narrative form. Other programs can use these topics as a guide.

Template 1. Program Context (All programs should complete #3 - #8)

1. Description of any state of institutional policies that may influence the application of program standards. 2. Description of the field and clinical experiences required for the program, including the number of hours for early field experiences and the number of hours/weeks for student teaching or internships. 3. Description of the criteria for admission, retention, and exit from the program, including required GPAs and minimum grade requirements for the content courses accepted by the program. 4. Description of the relationship of the program to the unit’s conceptual framework. 5. Indication of whether the program has a unique set of program assessments and the relationship of the program’s assessments to the unit’s assessment system. 6. Program of Study Sheet (outline of courses and experiences required to complete program). 7. Candidate and Completer Chart (provide three years of data, beginning with most recent). 8. Faculty Information (COE will collect and make this available to you electronically).

3 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 1a: Program of Study

Courses and Other Requirements Credits

Template 1b: Candidate and Completer Chart

Program Name

Academic Year Number of candidates enrolled in Number of program completers (Provide for last 3 years) the program 2007-2008 2006-2007 2005-2006

QuickTime™ and a decompressor The Unit (COE) are needed to see this picture. Assessment System

Our assessment system calls for each program to specify requirements at four transition points (entry, during program, completion, and follow-up) Requirements must align with the COE Conceptual Framework that calls for helping our students become knowledgeable, effective, and caring education professionals. Our conceptual framework matches the NCATE requirements to demonstrate evidence of students’ knowledge, skills, and dispositions. See the next section (Step 2: Align) for more information.

Unit (COE) Assessment System Program Requirements (Use specific, descriptive titles) Knowledgeable, Effective, Caring PROGRAMS Entry During Program Completion Follow-up

EDCS EDEA EDEF EDEP ETEC ITE Elementary ITE Secondary ITE MEdT KRS SPED

4 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Step 2. ALIGN Agree on what you are trying to accomplish

The first important alignment for COE programs is identifying requirements for all students at selected transition points in their program of study. The four transition points we use as a unit (college) are: 1. Entry (admission criteria) 2. During program (important assignments/projects during the program) 3. Completion (work required by the end of the program) 4. Follow-up (licensing exams; surveys of graduates, mentors, employers) COE uses a grid to show program requirements at the four transition points (template 2) aligned with the conceptual framework of helping our students become knowledgeable, effective, and caring professionals. Our framework aligns precisely with the NCATE requirement to show evidence of our students’ knowledge, skills and dispositions.

Template 2: Show Requirements at Transition Points (use specific, descriptive titles)

Entry During Program Completion Follow-up Your Program(s) Show separately for programs that have different assessments

The entry transition point consists of admissions criteria, such as applications, GPA, and interviews. The during program and completion transition points should include six to eight (not more than six to eight total) direct assessments of student work (e.g., projects, papers, presentations) that students complete to move from one program phase to the next. Follow-up consists of state licensing requirements, where applicable, and surveys of graduates, employers, and others who can provide program feedback.

As noted, the transition points should be aligned with the conceptual framework of helping students become knowledgeable, effective, and caring professionals, thereby demonstrating their knowledge, skills, and dispositions. Step 3 provides more information on selecting and writing assessments of student work.

In addition to specifying requirements at transition points, programs should identify their important outcomes or standards (i.e., professional, national, state), and show how they link with the program of study (courses and other requirements). Outcomes (often called student learner outcomes or SLOs) are what we want students to know, be able to do, and care about. Accrediting bodies often call these outcomes or standards, but terms such as objectives, goals, benchmarks, and performance indicators also are used in various contexts. Some accrediting bodies (e.g., NCATE SPAs) specify the standards and outcomes that programs must meet.

Don’t get sidetracked by terminology. Focus on what you want your students to learn and do. For COE, we must show links to the conceptual framework/KSD and any other national or state standards that apply to our programs (template 3).

5 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 3: Align Standards/Outcomes, Conceptual Framework, and Program of Study

Program Standards/Outcomes Other CFW Course Course Course Other What we want students to know, do, applicable & # # # professional KSD and care about standards 1.

You many want to use a simple check () to show the major emphases in your curriculum in Template 3. The Secondary Education Program uses a more detailed system to provide further information about various courses: I Introduce D Demonstrate A Apply to Practice P Portfolio Artifact

Template 3, which is a curriculum map or matrix, can help reveal gaps or overemphasis in various areas. Faculty members need time to work together to agree on the basic “must have” requirements for each course, while still leaving room for interests, talents, and creativity that individual faculty members bring to teaching and learning experiences. Think of the course must-have requirements as minimum qualifications for the course. The individuality that each instructor brings makes up the desirable qualifications. Good courses need both.

Some accrediting bodies, such as the Western Association of Schools and Colleges (WASC) require programs to show student learner outcomes in course syllabi. Faculty members should ensure that these outcomes are genuine and doable for themselves and their students. No one needs any more jargon these days.

Transformative; seamless; developmental; capstone; paradigm shift . . . Anyone? Anyone?

6 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Step 3. ASSESS Accumulate evidence

Next, faculty members must decide how to determine whether students really are learning what they think and hope students are learning. COE programs find that six to eight key assessments (e.g., papers, projects, assignments, exams, internship evaluations) provide the evidence they seek. Practical advice: Begin with assignments that students already complete as part of the program. Using assignments that are embedded in courses makes sense. Instructors or advisors who teach the courses or chair papers/projects can conduct the assessments, or programs might want to assign different or multiple assessors for students’ work. NCATE also requires COE to describe the processes we have adopted to ensure that assessment procedures are fair, accurate, consistent, and free of bias (template 6). The assessments you select to showcase your program (template 4) also go into the during program and completion transition points in Step 2 (template 2).

Template 4: List of 6-8 Key Program Assessments to demonstrate that students are knowledgeable, effective, and caring education professionals

Assessment Type or form of When assessment is Who assesses the assessment administered work 1 2 3 4 5 6

Once you select your assessments, another grid can help show whether all of your outcomes or standards are examined adequately in you plan (template 5).

Template 5: Relation of Program Standards/Outcomes (template 3) to the 6-8 Key Program Assessments (template 4)

Program Standards/Outcomes Assess- Assess- Assess- Assess- Assess- Assess- What we want students to know, do, ment ment ment ment ment ment 1 2 3 4 5 6 and care about 1

7 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 6: Processes to ensure that program assessment procedures are fair, accurate, consistent, and free of bias What processes has the program used to ensure that assessment procedures are fair, accurate, consistent, and free of bias? (What, when, how, and with what results?)

Further describing each assessment in detail is important. The College of Education uses the following format, (template 7) in accordance with NCATE SPA program review:

Template 7. Format for Writing Performance Assessments (do for EACH assessment)

Assessment 1: Title 1. Brief description of the assessment and its use in the program 2. Description of how the assessment aligns with specific standards 3. Brief analysis of data findings 4. Interpretation of how data provide evidence for meeting standards 5. Attachment of assessment documentation  Title  Type of assessment  When administered  Description  Standards  Instructions to students  Scoring guide (rubric, checklist, other criteria—see template 7a)  Student assessment data (tables, charts, graphs—see template 7b)

8 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Scoring guides consist of rubrics, checklists, or other criteria provided to (and sometimes co- developed with) students. Many programs in COE develop analytic rubrics (those that identify and score multiple components in a rubric, rather than providing only one holistic score). COE programs generally use three score points (unacceptable, acceptable, target) to match NCATE language. Instructors score each component of the rubric and provide an overall score.

Template 7a. Scoring Guide Format Assessment component Unacceptable Acceptable Target 1 Criteria . . . Criteria . . . Criteria . . . A T

U 2 Criteria . . . Criteria . . . Criteria . . . A T

U 3 Criteria . . . Criteria . . . Criteria . . . A T

U Overall score: U A T

Other ideas for scoring:

0—1—2

Does not meet expectations—Meets expectations—Exceeds expectations

Unsatisfactory—Satisfactory—Exemplary

Or, in the words of a group of middle school students:

Clueless—Getting it—Got it!

9 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Step 4. ANALYZE Aggregate, disaggregate—and aggravate just about everyone in the process

The need to collect and score six to eight assessments for many students across many programs (often within the same department) quickly raises the need for dependable technology tools to help manage the work. The College of Education is collaborating with other colleges and schools across campus to select a common set of technology tools that we all can use to collect and score student work, and to summarize the resulting data to suit our needs. COE has offered to “test drive” the technology tools that seem best suited to our campus as soon as they are available. COE plans to load each department’s assessments and scoring guides centrally from the Dean’s office. Students submit required work, and faculty members score the work with a radio-button or pull-down menu, similar to putting grades into Banner. The technology we are considering will allow each department to run its own reports by selecting descriptors of interest (e.g., assessment scores for all science methods courses taught online versus those taught on campus) from pull-down menus.

Programs need to report certain quantitative data for accreditation. For example, what number and percentage of students in various teacher education majors passed the required Praxis exams that HTSB requires for licensing? What number and percentage of students scored at the unacceptable, acceptable, and target levels on the six to eight key assessments selected by their programs? Data for program reports can be presented as shown in template 7b.

Template 7b. Student Assessment Data Format (make a table for each year—try to show 3 years of data) Assessment component Unacceptable Acceptable Target Number % N % N % 1 2 3 Overall score:

Departments and programs will need a variety of tables to aggregate and disaggregate data for different purposes. Data should be reported separately for different groups of students (Secondary BEd vs. post-bac) and various methods of program delivery (e.g., on-campus, off- campus, hybrid, online). For example, the Elementary and Early Childhood Education (EECE) program reports data separately for the Manoa and American Samoa programs. In the future reports, EECE also will report data separately for statewide distance education programs.

For NCATE purposes, programs should construct data tables to show scores for each component of an analytic rubric. Rather than showing multiple years of data in one table, a separate table for each year can be used to show results for all parts of the rubric. Simple-to- use, dependable technology tools are essential to preserve sanity during the aggregation and disaggregation process, and we are working on those!

10 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Step 5. ACT Apply what you find to help students learn better

Step 5 means using data to make programs better. Faculty members may be surprised by a particular result. The EECE program implemented an assessment to help determine the effects teacher candidates have on K-6 student learning. EECE learned that although the teacher candidates were quite skilled in reflecting on their own teaching performance, they were less able to plan next steps based on children’s performance. After recognizing this “self-reflection disorder,” the EECE faculty changed the program to ensure that teacher candidates have more opportunities to hone their assessment skills related to what children are learning.

COE programs that go through NCATE SPA review conclude with the following:

SECTION V—USE OF ASSESSMENT RESULTS TO IMPROVE CANDIDATE AND PROGRAM PERFORMANCE

Evidence must be presented in this section that assessment results have been analyzed and have been or will be used to improve candidate performance and strengthen the program. This description should not link improvements to individual assessments but, rather, it should summarize principal findings from the evidence, the faculty’s interpretation of those findings, and changes made in (or planned for) the program as a result. Describe the steps program faculty has taken to use information from assessments for improvement of both candidate performance and the program. This information should be organized around (1) content knowledge, (2) professional and pedagogical knowledge, skill, and dispositions, and (3) student learning. (Note: Knowledgeable, Effective, and Caring for COE)

Departments and programs can use the following guidelines to close the loop with regard to program improvement (template 9).

Template 8: Use of assessment results to improve candidate and program performance

1. Knowledgeable (knowledge)—What we want candidates to know (knowledge of discipline)  Principal findings and interpretation  Steps for improving candidate performance  Steps for improving program performance

2. Effective (skills)—What we want candidates to be able to do (theory ⇄ practice link; pedagogical content knowledge & skills; pedagogical & professional knowledge & skills)  Principal findings and interpretation  Steps for improving candidate performance  Steps for improving program performance 3. Caring (dispositions)—What we want candidates to care about (professional dispositions to help all students learn; verbal & nonverbal behaviors) *  Principal findings and interpretation  Steps for improving candidate performance  Steps for improving program performance * At a minimum, fairness and a belief that all students can learn

11 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

APPENDIX A Template Workbook (Available Electronically)

To be accompanied by making-sense-of-it-all narratives

Template 1. Program Context (All programs should complete #3 - #8)

Template 1a: Program of Study

Courses and Other Requirements Credits

Template 1b: Candidate and Completer Chart

Program Name

Academic Year Number of candidates enrolled in Number of program completers (Provide for last 3 years) the program 2007-2008 2006-2007 2005-2006

12 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 2: Show Requirements at Transition Points (use specific, descriptive titles)

Entry During Program Completion Follow-up Your Program(s) Show separately for programs that have different assessments

Template 3: Align Standards/Outcomes, Conceptual Framework, and Program of Study

Program Standards/Outcomes Other CFW Course # Course # Course # Course # What we want students to know, do, applicable & professional KSD and care about standards 1.

Template 4: List of 6-8 Key Program Assessments to demonstrate that students are knowledgeable, effective, and caring

Assessment Type or form of When assessment is Who assesses the assessment administered work 1 2 3 4 5 6

13 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 5: Relation of Program Standards/Outcomes (template 3) to the 6-8 Key Program Assessments (template 4)

Program Standards/Outcomes Assess- Assess- Assess- Assess- Assess- Assess- What we want students to know, do, ment ment ment ment ment ment 1 2 3 4 5 6 and care about 1

Template 7. Format for Writing Performance Assessments (do for EACH assessment)

14 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 7a. Scoring Guide Format (do for EACH assessment) Assessment component Unacceptable Acceptable Target 1 Criteria . . . Criteria . . . Criteria . . . U A T 2 Criteria . . . Criteria . . . Criteria . . . U A T 3 Criteria . . . Criteria . . . Criteria . . . U A T Overall score: U A T

Template 7b. Student Assessment Data Format (make a table for each year—try to show 3 years of data) Year: 2007-08 Unacceptable Acceptable Target Assessment component Number % N % N % 1 2 3 Overall score:

Year: 2006-07 Unacceptable Acceptable Target Assessment component Number % N % N % 1 2 3 Overall score:

Year: 2005-06 Unacceptable Acceptable Target Assessment component Number % N % N % 1 2 3 Overall score:

15 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 8: Use of assessment results to improve candidate and program performance

1. Knowledgeable (knowledge)—What we want candidates to know (knowledge of discipline)

 Principal findings and interpretation  Steps for improving candidate performance  Steps for improving program performance

2. Effective (skills)—What we want candidates to be able to do (theory ⇄ practice link; pedagogical content knowledge & skills; pedagogical & professional knowledge & skills)

 Principal findings and interpretation  Steps for improving candidate performance  Steps for improving program performance

3. Caring (dispositions)—What we want candidates to care about (professional dispositions to help all students learn; verbal & nonverbal behaviors) *

 Principal findings and interpretation  Steps for improving candidate performance  Steps for improving program performance

* At a minimum, fairness and a belief that all students can learn

Continuous program improvement

Good accreditation reports, too!

16 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

ADDITIONAL UNIT QUESTIONS FOR STANDARD 2

STANDARD 2. ASSESSMENT SYSTEM AND UNIT EVALUATION

The unit has an assessment system that collects and analyzes data on the applicant qualifications, the candidate and graduate performance, and unit operations to evaluate and improve the unit and its programs. [Note: Include programs for teachers, including graduate programs for licensed teachers, and other school professionals, noting differences when appropriate.] 2a. Assessment System

1. How is the unit assessment system evaluated and continuously improved? Who is involved and how?

2. Please complete the following table (Table 6) to indicate the key assessments used by the unit and its programs to monitor candidate performance at transition points such as those listed in Table 6?

Table 6 Unit Assessment System: Transition Point Assessments (We will use Template 2 on page 13 in this handbook)

Template 2: Show Requirements at Transition Points (use specific, descriptive titles) Entry During Program Completion Follow-up Your Program(s) Show separately for programs that have different assessments

3. How does the unit ensure that the assessment system collects information on candidate proficiencies outlined in the unit's conceptual framework, state standards, and professional standards?

4. How does the unit ensure its assessment procedures are fair, accurate, consistent, and free of bias? (We will use Template 6 on page 14 of this handbook.)

5. What assessments and evaluations are used to manage and improve the operations and programs of the unit?

6. (Optional) One or more tables and links to key exhibits related to the unit assessment system could be attached here. Data in tables should be discussed in the appropriate prompt of 2a. (Links with descriptions must be typed into a Word document that can be uploaded here. The number of attached exhibits should be limited in number; BOE members should access most of the exhibits in the unit's electronic exhibit room.)

17 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

2b. Data Collection, Analysis, and Evaluation

1. What are the processes and timelines used by the unit to collect, compile, aggregate, summarize, and analyze data on candidate performance, unit operations, and program quality?

 How are the data collected?  How often are the data summarized and analyzed?  Whose responsibility is it to summarize and analyze the data? (Dean, assistant dean, data coordinator, etc.)  In what formats are the data summarized and analyzed? (Reports, tables, charts,  graphs, etc.)  What information technologies are used to maintain the unit's assessment system?

2. How does the unit maintain records of formal candidate complaints and their resolutions?

3. (Optional) One or more tables and links to key exhibits related to the data collection, analysis, and evaluation could be attached here. Data in tables should be discussed in the appropriate prompt of 2b. (Links with descriptions must be typed into a Word document that can be uploaded here.) 2c. Use of Data for Program Improvement

1. What are assessment data indicating about candidate performance on the main campus, at off-campus sites, and in distance learning programs?

2. How are data regularly used by candidates and faculty to improve their performance?

3. How are data used to discuss or initiate program or unit changes on a regular basis?

4. What data-driven changes have occurred over the past three years?

5. How are assessment data shared with candidates, faculty, and other stakeholders?

6. (Optional) One or more tables and links to key exhibits related to the use of data for program improvement could be attached here. Data in tables should be discussed in the appropriate prompt of 2c. (Links with descriptions must be typed into a Word document that can be uploaded here. The number of attached exhibits should be limited in number; BOE members should access most of the exhibits in the unit's electronic exhibit room.) Optional

1. What does your unit do particularly well related to Standard 2?

2. What research related to Standard 2 is being conducted by the unit?

18 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

COE Program Examples

Elementary and Early Childhood Education Program

Template 1. Program Context 1. Description of any state or institutional policies that may influence the application of SPA standards.

The State of Hawai‘i Teacher Standards Board (HTSB) developed the Teacher Performance and Licensing Standards, referred to in this document as the Hawai‘i Teacher Standards (HTS), to set standards for Hawai‘i’s public school teachers. The standards are aligned with those of NCATE, the Interstate New Teacher Assessment and Support Consortium (INTASC), and the National Board for Professional Teaching Standards (NBPTS). The requirements of the Hawai‘i Teacher Standards Board align fully with the requirements of the Association for Childhood Education International (ACEI) in the application of SPA standards. Teacher candidates must demonstrate their knowledge, skills, and dispositions as knowledgeable, effective, and caring teachers to meet each of the ACEI and Hawai‘i Teacher Standards. The EECE program illustrates the relationship between the ACEI standards and the HTS (in italics) as follows:

Elementary and Early Childhood Education (EECE) Program

Demonstrating the Knowledge, Skills, and Dispositions of Knowledgeable, Effective, and Caring Teachers

Association for Childhood Education International (ACEI) I. Development, Learning, and Motivation Hawai‘i Teacher Standards (HTS) 1. Focuses on the Learner 2. Creates and Maintains a Safe and Positive Learning Environment 4. Fosters Effective Communication in the Learning Environment

Association for Childhood Education International (ACEI) II. Curriculum Hawai‘i Teacher Standards (HTS) 5. Demonstrates Knowledge of Content

Association for Childhood Education International (ACEI) III. Instruction Hawai‘i Teacher Standards (HTS) 3. Adapts to the Learner 6. Designs and Provides Meaningful Learning Experiences 7. Uses Active Student Learning Strategies

Association for Childhood Education International (ACEI) IV. Assessment Hawai‘i Teacher Standards (HTS) 8. Uses Assessment Strategies

Association for Childhood Education International (ACEI) V. Professionalism Hawai‘i Teacher Standards (HTS) 9. Demonstrates Professionalism 10. Fosters Parent and School Community Relationships

19 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

2. Description of the field and clinical experiences required for the program, including the number of hours for early field experiences and the number of hours/weeks for student teaching or internships.

Teacher candidates participate in three semesters of structured field experience (two days per week) and one semester of student teaching (five days per week) for a combined experience of more than 1200 hours in K-6 partner school classrooms. Candidates participate in field experiences in lower and upper elementary grades in diverse partner schools selected for each cohort group.

Settings: Partner schools are selected through the collaborative efforts of the EECE placement coordinator, faculty cohort coordinators, DOE complex area superintendents, principals, and K-6 mentor teachers. The EECE placement coordinator ensures that candidates have access to diverse K-6 settings that vary in terms of children’s socioeconomic level, culture, race/ethnicity, academic performance, and community (e.g., students from military families stationed in Hawai‘i).

Experiences: Expectations for field experience are described fully in the EECE Program Handbook and in the ITE 317 and 390 course syllabi provided on the EECE website at www.hawaii.edu/coe/departments/ite/prog_bed_elem.html. To illustrate, expectations for field experience in semester 1 are summarized as follows. Candidates will:

Overall goal—Build positive relationships with their K-6 students and mentor teachers, learn about the children in their classrooms, and help create a safe and engaging learning environment. Teach at least two lessons without assistance by the end of the semester.

1. Demonstrate professionalism by being on time, staying the entire day, assisting with mentor teacher’s schedule (e.g., recess duty), being well prepared for school, providing written lesson plans in advance, and maintaining confidentiality related to students and colleagues. 2. Attend professional meetings (faculty meetings, professional development seminars, parent and/or student conferences, IEP meetings) within field experience days as specified by mentor teachers. Attend all field experience seminars with cohort. 3. Build positive relationships with mentor teachers, K-6 students, peers, school staff, and parents/caregivers. 4. Take initiative to assist mentor teachers (e.g., work with individuals and small groups, read a story, help students with self-assessment, prepare lesson materials). 5. Participate in and take increasing responsibility for leading routines, business, other activities, and transitions. 6. Take responsibility for ongoing communication with mentor teachers and UHM supervisors, as specified for the semester (e.g., conferences, journals, logs). 7. Observe and reflect on classroom management, planning, teaching, assessment, and other professional practices, in relation to COE framework. 8. Observe and reflect on K-6 students’ interactions in relation to the Hawai‘i DOE General Learner Outcomes (GLOs) and Hawai‘i DOE Content and Performance Standards III (http://doe.k12.hi.us/). 9. Teach and reflect on lessons required in methods courses and as specified by mentor teachers and UHM field supervisors. Integrate technology as appropriate. 10. Interact and meet with parents and caregivers whenever possible. 11. Prepare self-assessment evidence for mid-semester and end-of-semester evaluation, and set up conferences with mentor teacher. 12. Begin building professional portfolio to provide evidence of meeting Hawai‘i Teacher Standards throughout the semester, including evidence of student learning (student work, pictures, movies, other electronic work).

20 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Candidates assume increasing responsibility over the course of the four-semester program, leading to a three-week solo teaching period during student teaching. Duration of experiences: Semester 1 15 weeks 2 days/week 7 hours/day 210 hours Semester 2 15 weeks 2 days/week 7 hours/day 210 hours Semester 3 15 weeks 2 days/week 7 hours/day 210 hours Semester 4 15 weeks 5 days/week 8 hours/day 600 hours

Criteria for selecting sites:  Partnership school commits to at least two years of service. During semesters 1, 2, and 3, candidates spend 2 days/week working in the field. During semester 4 (student teaching) candidates spend 5 days/week working in the field.  Partnership school commits to providing at least four mentors each semester.  Partnership school commits to providing teacher candidates with opportunities to practice teaching through implementing lessons developed through candidate course work and in conjunction with mentor teachers.

Partnerships are initiated upon recommendation from the DOE complex area superintendents (CAS). The CAS provides a list of recommended schools to the EECE placement coordinator and facilitates contacts with principals. The placement coordinator then facilitates meetings between principals and EECE cohort coordinators.

Criteria for selecting and training mentor teachers: Mentor teachers commit to providing candidates with feedback and periodic formal assessments during mid-semester and end-of-semester evaluation conferences. The EECE placement coordinator works with principals to secure agreements to host EECE candidates. The EECE placement coordinator, in turn provides the list of available schools to cohort coordinators. Cohort coordinators meet with mentor teachers to discuss participation. Mentors must be tenured teachers who agree to share their practice and classroom with teacher candidates, work and talk with them to provide guidance and the benefit of their experience, and allow candidates to gradually increase their teaching participation in a safe environment. Mentor teachers are trained through bi-annual on-campus sessions and through ongoing small group and individual meetings arranged by cohort coordinators.

Criteria for selecting and training faculty supervisors: Faculty field supervisors in the EECE program are full- and part-time faculty members, in both tenure- track and limited-term appointments. All are required to have had experience teaching in K-6 classrooms and working with pre-service teachers. Faculty supervisors are trained through pre-semester workshops, faculty meetings, and regular meetings with cohort coordinators.

3. Description of the criteria for admission, retention, and exit from the program, including required GPAs and minimum grade requirements for the content courses accepted by the program.

The Elementary and Early Childhood Education program qualifies graduates to teach in elementary schools in grades K–6. Requirements include completion of the General Education Core specified for education majors, elementary emphasis and methods courses in elementary education, and student teaching, for a minimum of 124 credit hours. Upon admission, students are assigned to cohort groups and enroll in a set sequence of courses over four semesters.

Admission Criteria  A minimum cumulative grade point average of 2.75 from all post-secondary institutions attended.  A minimal composite score of 516 in the reading, writing, and mathematics subtests of the Pre-Professional Skills Test (PPST) or Computerized Pre-Professional Skills Test (C-PPST) of the PRAXIS series with a minimum score of 170 in each subtest.  A minimum of 40 documented hours of current (within the past five years) group leadership involvement with elementary-aged children. Such field experience(s) may be completed on either a paid or voluntary basis. It is highly recommended that the field experience(s) span a period of at least

21 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

two semesters.  Oral and non-verbal communicative competence demonstrated through the successful completion of a personal admissions interview. The purpose of the interview is to assess fluency and intelligibility of oral communication, interest in teaching, student motivation, and suitability to the profession.  Completion of a small group interview with the faculty of the Elementary and Early Childhood Education Program (and faculty of the Special Education Department for candidates in dual preparation).

Retention Criteria  Minimum GPA of 2.75.  Grades of C or better in all program courses (a grade of C- requires repeating the course to remain in the program).  Successful completion of field experience, indicated by scores of Acceptable or Target on all Hawai‘i Teacher Performance Standards and EECE Professional Dispositions for the semester’s work, or successful completion of a formal Plan of Assistance for Improvement, specified through collaboration of UHM faculty, mentor teachers, and teacher candidate.

Program Exit Criteria  Minimum GPA of 2.75.  Successful completion of student teaching, indicated by scores of Acceptable or Target on all Hawai‘i Teacher Performance Standards and EECE Professional Dispositions, or successful completion of a formal Plan of Assistance for Improvement, specified through collaboration of faculty, mentor teachers, and teacher candidate.

4. Description of the relationship of the program to the unit’s conceptual framework.

The conceptual framework of the unit states that the College of Education consists of educators who provide innovative and cutting-edge research and teaching in an effort to further the field of education and prepare educators to contribute to a just and democratic society. The mission is to work as a diverse and democratic community to prepare new educators and provide ongoing professional development in education; increase the knowledge base in education and related fields through the production and application of research related to teaching, learning, and assessment; and serve as partners and leaders for excellence in education. The philosophy states that the COE is committed to preparing education professionals who are knowledgeable, effective, and caring, and who demonstrate those core values through their (k)nowledge, (s)kills, and (d)ispositions in ways specified in depth in the conceptual framework.

The EECE program is fully aligned with the COE conceptual framework. EECE faculty representatives participated fully in the revision and updating of the COE conceptual framework. To reflect and build upon the COE conceptual framework, the EECE faculty members added the following to the EECE Program Handbook and syllabi:

The College of Education Conceptual Framework provides the organizing system for documenting professional growth in the EECE program. The vision of the college is to prepare educators to contribute to a just and democratic society. The college and the EECE program require teacher candidates to meet the standards or core values of being knowledgeable, effective, and caring. The EECE program describes the standards with these Hawaiian proverbs selected from ‘Olelo No‘eau (Pukui, 2001):

Knowledgeable  Teacher candidates are knowledgeable about content, pedagogy, and professionalism; human growth and development; and the physical, mental, emotional, and social needs of students with diverse backgrounds and learning needs.  E lawe i ke a‘o malama, a e ‘oi mau ka na‘auao: He who takes his teachings and applies them increases his knowledge (‘Olelo No‘eau 328).  ‘A‘ohe pau ka ‘ike i ka halau ho‘okahi: All knowledge is not taught in the same school. One can learn from many sources (‘Olelo No‘eau 203). 22 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Effective  Teacher candidates have the professional experience and skills to teach so that all children can learn.  Ho a‘e ka ‘ike he‘enalu i ka hokua o ka ‘ale: Show [your] knowledge of surfing on the back of the wave. Talking about one’s knowledge and skill is not enough; let it be proven (‘Olelo No‘eau 1013).

Caring Teacher candidates care about students and their families and communities, teaching and learning, and their own professional development.  E kuahui like i ka hana: Let everybody pitch in and work together (‘Olelo No‘eau 323).

The core values knowledgeable, effective, and caring are the basis for the EECE Professional Dispositions and are specified in relation to the Hawai‘i Teacher Standards that teacher candidates must meet for successful program completion.

5. Indication of whether the program has a unique set of program assessments, which are different from those expected for the unit’s assessment system, and the relationship of the program’s assessments to the unit’s assessment system.

The College (unit) assessment system is consistent with the conceptual framework and collects, organizes, and analyzes data about applicant qualifications, candidate performance, and unit operations in a consistent and systematic manner. Each preparation program has a comprehensive and integrated assessment plan that comprises evaluative measures designed to inform decisions about program change. The core of the unit’s assessment system is evaluation of candidate performance using multiple assessments at key transition points. Teacher candidates develop course products and work samples, which are submitted and reviewed at specified points.

The Elementary and Early Childhood Education Program assessment system includes both unit and program specific assessments. The unit and program assess candidate performance at entrance, midpoint/prior to student teaching, and exit. The following is a summary of the EECE assessment system. Entry  Application  Interviews (Advising / EECE Faculty)  Grades/GPA ≥2.75  Praxis I scores  Pre-education Core or AA degree  Documented involvement with K-6 children Midpoint (end of year 1)  Grades/GPA ≥ 2.75  Grades of C or better in all courses  Courses completed (ITE 312, 317 Field Experience, and Methods)  Acceptable/Target score on content-based assessment of methods course grades  Acceptable/Target scores on lesson planning assessment across methods courses  Acceptable/Target Dispositions score  Acceptable/Target Field Experience Evaluation score  Acceptable/Target Professional Portfolio developmental score Exit (end of year 2)  Grades/GPA > 2.75  Grades of C or better in all courses  Praxis II scores  Courses completed (ITE 317 Field Experience, Methods, and Student Teaching ITE 390/391)  Acceptable/Target score on content-based assessment of methods course grades  Acceptable/Target scores on lesson planning assessment across methods courses  Acceptable/Target Dispositions score  Acceptable/Target Student Teaching Evaluation score 23 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

 Acceptable/Target Student Teaching Work Sample (analysis of K-6 student learning)  Acceptable/Target Professional Portfolio final score  Student Teaching Survey  Institutional Recommendation Post Graduate  State License Exam (Praxis II), Employer Surveys, Graduate Surveys Template 1a: Program of Study

Courses and Other Requirements Credits Elementary Education Emphasis (18 credits)  EDEF 310 Education in American Society 3  EDEP 311 Psychological Foundations 3  etc. etc. Professional Education Core (51 credits)  ITE 312 Introduction to Teaching, Elementary 3  ITE 313 Literature & Literacy I 3  etc etc

Template 1b: Candidate and Completer Chart

Program Name

Academic Year Number of candidates enrolled in Number of program completers (Provide for last 3 years) the program 2003-2004 296 211 2004-2005 325 129 2005-2006 336 173

Template 2: Show Requirements at Transition Points (use specific, descriptive titles)

EECE Entry During Program Completion Follow-up  Application  GPA > 2.75  GPA > 2.75  State  Pre-  ITE 312  ITE 390 (student licensing exam education core or (Introduction to Teaching, teaching) and ITE 391  Follow-up AA/AAT ITE 317 (field experience), (seminar) survey of  Documented and methods courses  Analysis of K-6 graduates, work with K-6  Lesson Planning Student Learning mentors, and children Across the K-6 Curriculum  Summary employers  Praxis I  Resource Kit for Reflection Paper for (PPST Reading, Working with Children with Professional Teaching Writing, Disabilities Portfolio Mathematics)  Planning and  Final  Cumulative Designing Instruction: A Assessment of Student 2.75 GPA Technology Integrated Teaching  Small group Learning Unit  Score of interview with  Score of acceptable acceptable or target on EECE faculty or target on dispositions dispositions  Student teaching completion survey

24 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 3: Align Standards/Outcomes, Conceptual Framework, and Program of Study

ACEI HTPS CFW ITE ITE ITE ITE ITE ITE ITE ITE ITE Standards & 312 313 317 322 324 326 343 390 391 KSD & & & & 314 323 325 329 Development, 1, 2, 4 K     learning, and motivation Curriculum 5 K, E        

Instruction 3,6,7 K, E        

Assessment 8 K, E        

Professional- 9, 10 C     ism

ACEI Association for Childhood Education International ITE 312 Introduction to Teaching, Elementary (NCATE SPA Standards for Elementary Education) ITE 313 Literature & Literacy I HTPS Hawai‘i Teacher Performance Standards ITE 314 Literature & Literacy II KSD Knowledge, skills, and dispositions ITE 317 Field Experience CFW COE Conceptual Framework (knowledgeable, ITE 322 Social Studies effective, caring) ITE 323 Science ITE 324 Mathematics I ITE 325 Mathematics II ITE 326 Visual Arts ITE 329 Performing Arts Expression ITE 343 Personal & Social K-6 Health Skills ITE 390 Student Teaching ITE 391 Seminar in Student Teaching

Template 4: List of 6-8 Key Program Assessments to demonstrate that students are knowledgeable, effective, and caring Assessment Type of assessment When administered Who assesses the work 1. Licensure State licensure exams Semester 3 or 4 Hawai‘i Teacher Standards Assessment (Praxis II) Board (licensure) and EECE faculty (program improvement) 2. Methods Course Earned letter grades in Semesters 1, 2, 3: Methods course instructors Grades nine required methods ITE 313, 314, 322, courses 323, 324, 325, 326, 329, 343 3. Lesson Planning Lesson plans, teaching, Semesters 1, 2, 3: Methods course instructors Across the K-6 reflection ITE 314, 322, 323, Curriculum 325, 326, 329, 343 4. Final Evaluation of End of semester Semester 4: Student Mentor teachers and Student Teaching assessment of Hawai‘i Teaching, ITE 390 university field supervisors Teaching Standards and EECE Professional Dispositions 5. Analysis of K-6 Teacher work sample Semester 4: Student Cohort Coordinators Student Learning Teaching Seminar, ITE 391 6. Summary Written summary Semester 4: Student Cohort Coordinators Reflection Paper for reflection document Teaching Seminar, Professional Teaching ITE 391 Portfolio 7. Resource Kit for Resource kit for general Semester 2 or 3: SPED course instructors Working with Children education teachers SPED 444 with Disabilities 8. Planning and Learning unit Semester 2 or 3: ETEC course instructors 25 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Designing Instruction: A ETEC 414, 442, 447, Technology Integrated 448, or 480 Learning Unit`

Template 5: Relate Program Standards/Outcomes to the 6-8 Assessments in Template 4

ACEI Standards 1 2 3 4 5 6 7 8 1. Development, Learning, and Motivation        2. Curriculum Standards 2.1 English language arts       2.2 Science       2.3 Mathematics       2.4 Social studies       2.5 The arts      2.6 Health education      2.7 Physical education      2.8 Connections across curriculum      3. Instruction Standards 3.1 Integrating and applying knowledge for instruction        3.2 Adaptation to diverse students        3.3 Development of critical thinking, problem solving, performance skills       3.4 Active engagement in learning      3.5 Communication to foster collaboration      4. Assessment for Instruction       5. 1 Practices and behaviors of developing career teachers        5.2 Reflection and evaluation        5.3 Collaboration with families    5.4 Collaboration with colleagues and the community   

26 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 6: Processes to ensure that program assessment procedures are fair, accurate, consistent, and free of bias

What processes has the program used to ensure that assessment procedures are fair, accurate, consistent, and free of bias? (What, when, how, and with what results?) EECE faculty members want to give more attention to improving inter-rater reliability and reducing subjectivity in assessment. This change will involve refining our assessments to make them as clear as possible for instructors, mentors, and candidates while also providing anchor papers and exemplars to illustrate candidate performance. The program formed a committee to examine all assessment procedures from the perspective of fairness, accuracy, consistency, and freedom from bias. The committee will work with faculty to make modifications and improvements in the assessment system in the coming academic year.

At the end of Spring 2007, the Program Assessments and Evaluation committee deferred reliability and validity studies of ACEI assessments because the rubrics used to assess the student work samples were revised and would be significantly different from those that would be used in 2008.

At the end of Spring 2008, faculty members collected and assessed, using common rubrics, three student work samples representing evidence of the students’ knowledge, skills, and dispositions (ACEI #3, #5, & #6 projects). The students’ pieces were uploaded and assessed in the TK-20 data storage system.

Fall 2008: The Program Assessments and Evaluation committee will be examining the reliability of raters’ estimates of our students’ performance on the ACEI #3, #5, & #6 projects. We might look at the reliability as a type of cross-tab (chi-square) table. Ideally, we would like raters to agree on every rating. An examination of the errors (i.e., the inconsistency in ratings) would provide information about interrater reliability. We can determine how much of the error is likely systematic (one rater rates everything easier or more difficult) or whether other rater errors might be more likely to interact with the task, after controlling for chance agreement.

27 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 7. Format for Writing Performance Assessments (do for each assessment)

EECE Assessment 5: Analysis of K-6 Student Learning 1. Brief description of the assessment and its use in the program During student teaching, candidates select a minimum of three samples of student work related to a specific lesson or series of lessons to illustrate how students learned the content and skills addressed in the lesson. The student work samples must show a range of responses (unacceptable, acceptable, and target, as applicable). The narrative analysis should provide reflections on teacher strengths and areas for improvement in planning, teaching, and assessment in relation to student work and plans for next steps.

2. Description of how the assessment specifically aligns with specific standards This assessment aligns with ACEI Standards 1, 2.1-2.4, 3.1-3.4, 4, and 5.1-5.2. Teacher candidates are assessed on the assessment strategies they select in relation to the standards and benchmarks they are teaching, their selection of student work to illustrate the range of student learning in the classroom, their reflective analysis, and use of data to plan next steps. Candidates analyze K-6 student work to find evidence of student learning and to improve their own planning, assessment, and teaching. The teacher work sample must contain the following elements:  Description of the classroom, school, and community context [ACEI #1]  Rationale (why the lessons were chosen, significance of the topic to students) [ACEI #1, 2.1- 2.4, 3.1-3.4]  Standards and benchmarks to be addressed in the lessons [ACEI #1, 2.1-2.4, 3.1-3.4]  Assessment procedures (formative and summative) [ACEI #4]  Instructional plans (teaching strategies that engage students) [ACEI #3.1-3.4]  Student work display and interpretation [ACEI #4]  Reflection and next steps [ACEI #4, 5.1-5.2]

3. Brief analysis of data findings Rubric scores for this assessment indicate that analyzing K-6 student work and making decisions based on that analysis is one of the most challenging areas for our teacher candidates. Almost all candidates scored at the acceptable or target level, but scores were split more evenly across the two levels than in other program assessments. Some candidates did not demonstrate acceptable scores on this assessment, particularly in the area of using data for planning next steps. These findings are consistent with the analysis of the data from Assessment 4. Student teachers were rated more often as acceptable in this category than in other categories. The EECE faculty members recognize that assessment is an area in which our candidates need more in-depth preparation. We have begun planning to address that need specifically this year, as discussed in Section V—Use of Assessment Results to Improve Candidate and Program Performance.

4. Interpretation of how data provide evidence for meeting standards This assessment was a new one for our program in the 2005-06 academic year and one that requires candidates to take their learning full circle. The data indicate that our teacher candidates are able to draw conclusions about their effect on K-6 student learning based on their lesson design, teaching, assessment, reflective analysis, and use of data. However, this assessment also provides evidence that the EECE program must structure earlier and more frequent opportunities for candidates to engage in designing assessments matched with content and learning goals, analyzing student work, and, in particular, making instructional plans for individuals and the class as a whole based on their analysis. We need to provide more guidance and practice in the varied uses of assessment related to children’s learning and to candidates’ own professional growth. The data show that while most students achieved acceptable or target ratings for Assessment 5, there is considerable room for growth in this area as compared to any other assessment. Our candidates are at a beginning stage as new educators in this aspect of teaching.

28 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 7. Format for Writing Performance Assessments (continued)

Attachment of Assessment Documentation Title: Analysis of K-6 Student Learning

Type of assessment: Teacher work sample

When administered: Semester 4 (student teaching)

Description: Teacher candidates select a minimum of three samples of student work related to a specific lesson or series of lessons to illustrate how students learned the content and skills addressed in the lesson. The student work samples should show a range of responses (unacceptable, acceptable, and target, as applicable). The narrative analysis should provide reflections on teacher strengths and areas for improvement in planning, teaching, and assessment in relation to student work and plans for next steps.

Standards: 1. Development, learning, and motivation 2. Curriculum (2.1, 2.2, 2.3, 2.4) 3. Instruction (3.1, 3.2, 3.3, 3.4) 4. Assessment 5. Professionalism (5.1, 5.2)

Instructions to teacher candidates: Teacher candidates prepare a teacher work sample to demonstrate their knowledge and understanding of the student learning in their classrooms. Teacher candidates select a minimum of three samples of student work related to a specific lesson or series of lessons to illustrate how students learned the content and skills addressed in the lesson. The teacher work sample should contain the following elements:  Description of the classroom, school, and community context  Rationale (why the lessons were chosen, significance of the topic to students)  Standards and benchmarks to be addressed in the lesson(s)  Assessment procedures (formative and summative)  Instructional plans (teaching strategies that engage students)  Student work display and interpretation  Reflection and next steps Teacher candidates are assessed on the assessment strategies they select in relation to the standards and benchmarks they are teaching (25%), their selection of student work to illustrate the range of student learning in the classroom (20%), their reflective analysis (30%), and use of data to plan next steps (25%).

29 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 7a. Scoring Guide Format

Scoring Guide for Assessment 5

Component Unacceptable Acceptable Target Assessment  Candidates do not  Candidates plan  Candidates plan strategies specify assessment assessment strategies authentic assessment 25% strategies or use aligned with content, strategies aligned with ACEI 1, 2.1- strategies not connected learning opportunities, content, learning 2.4, 4 to content or students. and knowledge of opportunities, and  Candidates use students. knowledge of students, assessment as an  Candidates plan and to provide afterthought, rather than formal and informal opportunities for students’ as an integral part of assessment strategies self-assessment. lesson planning. to gather information  Candidates plan about teaching and specific formal and learning. informal assessment strategies to frame decisions about teaching and learning.

Unacceptable  Acceptable  Target  Selection of  Candidates do not  Candidates  Candidates select student work select student work or select student work that student work that samples work cannot be used to represents the range of represents the range of 20% assess learning (e.g., performance in the performance in the class, ACEI 4 work does not match class (unacceptable, and makes connections standards or assessment acceptable, and target betweens students’ plans). levels, as applicable). previous and current performance.

Unacceptable  Acceptable  Target  Reflective  Candidates do not  Candidate  Candidates compare analysis analyze student work or analysis of student and contrast student 30% reflection does not work indicates learning to plan upcoming ACEI 1, 4, provide direction for understanding of instruction. 5.1-5.2 teaching and learning. student learning.  Candidate reflection  Candidate indicates strategies reflection indicates targeted to assist what teacher should individual student do next. learning.

Unacceptable  Acceptable  Target  Use of data  Candidates do not  Candidates use  Candidates use data 25% use data or use data data to plan for to modify instruction to ACEI 1, 4, only to describe individual learning and promote individual 5.1-5.2 students. development. learning and development.  Candidates consult with colleagues and specialists as indicated.

Unacceptable  Acceptable  Target 

Overall Unacceptable  Overall Acceptable  Overall Target 

30 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 7b. Student Assessment Data Format (for each year—try to show 3 years)

Student Assessment Data for EECE Assessment 5

Spring 2006 New Assessment (one semester of data) Manoa Component Unacceptable Acceptable Target Total Passing Number % Number % Number % Number % Assessment 1 0.01% 64 50% 63 49.2% 127 99.2% strategies (25%) Selection of student 1 0.01% 62 48.4% 65 50.8% 127 99.2% work samples (20%) Reflective analysis 2 0.02% 64 50% 62 48.4% 126 98.4% (30%) Use of data (25%) 12 0.09% 70 54.7% 46 35.9% 116 90.6% Overall 2 0.02% 69 53.9% 57 44.5% 126 98.4%

American Samoa Component Unacceptable Acceptable Target Total Passing Number % Number % Number % Number % Assessment 1 5.88% 10 58.82% 6 35.30% 16 94.12% strategies (25%) Selection of student 0 0% 10 58.82% 7 41.18% 17 100% work samples (20%) Reflective analysis 1 5.88% 12 70.59% 4 23.53% 16 94.12% (30%) Use of data (25%) 2 11.76% 10 58.82% 5 29.42% 15 88.24% Overall 1 5.88% 12 70.59% 4 23.53% 16 94.12%

31 University of Hawai‘i at Mānoa College of Education [Dr. Beth Pateman] 8/11/08

Template 8: Use of assessment results to improve candidate and program performance Example page from EECE SPA Report III. Effects on Student Learning and on Creating Environments that Support Learning Principal Findings and Interpretation Results from assessments related to effects on student learning were the biggest eye-opener for program faculty in this report. We learned that our candidates were stronger in planning and implementing lessons than in aligning and using the results of assessment to improve K-6 student learning. Our candidates performed at acceptable and target levels, but we see tremendous need for program growth and candidate development in these areas. As we have reviewed our data, faculty members have commented that we must help candidates understand and apply different kinds of assessment. Candidates are able to focus on what they did and how they could have implemented their lessons better. They are less able to reflect on the quality of work that children did and what that implies about students and, thus, their instruction. As one faculty member said, we need to help our candidates come full circle—from learning where children are before beginning lessons to understanding where both they and the children have come in the process. We want our candidates to focus on multiple indicators of student learning, including affective responses to implemented lessons.

Improving Candidate Performance Candidates need earlier and more frequent opportunities in semesters 2 and 3 to engage in designing assessments matched with content and learning goals, analyzing student work, and, in particular, making instructional plans for individuals and the class as a whole based on their analysis. We need to provide more guidance and practice in the varied uses of assessment and reflection related to children’s learning and, in turn, to candidates’ growth. Faculty members need to provide candidates with anchor papers and exemplars of quality candidate work to assist them in preparing their own work for our assessment system. Our faculty members need to plan the best ways of doing this so that we provide thoughtful guidance rather than a directive, cookie-cutter approach to planning and assessment.

Improving the EECE Program The EECE faculty members are in discussion about further improving the preparation of our general elementary education candidates in working with children and families with special needs. We partner closely with the Department of Special Education in our college to offer an additional emphasis in Dual Preparation (preparation for a dual teaching license in Elementary Education and Special Education) for candidates who select this option. However, we believe that all candidates would benefit from additional preparation. The courses we are considering for all candidates include SPED 425 Partnerships with Families and Professionals and SPED 461 Assessment, Planning, and Instruction for Students with Mild/Moderate Disabilities. The EECE faculty members also need to refine the performance indicators that form the rubrics for our assessment system. The past year was a time of bringing our assessment system to life after several years of planning, revision, and refinement. Now that the system is in motion, we must ensure that we provide increasingly clear definitions and understandable expectations for candidates in their coursework and field experiences. We also must ensure that all stakeholders—candidates, mentors, administrators, faculty, and field supervisors—are engaged in the refinement process.