2005-06 Pilot Year Score Report All Institutions
Total Page：16
File Type：pdf, Size：1020Kb
Performance Assessment for California Teachers – Pilot Year 4 (2005-06) iii. Sample Score Report This example from PACT Central is a consortium-wide score report from the 2005-06 pilot year.
2005-06 PILOT YEAR SCORE REPORT– ALL INSTITUTIONS
Table 1. Mean Item Scores by Subject Area – 2005-06 Pilot Year Planning Instruction Assessmen Reflection Academic Total MIS MIS MIS t MIS MIS Language (Std. Dev.) (Std. Dev.) (Std. Dev.) (Std. Dev.) (Std. Dev.) (Std. Dev.) Mean 2.557 2.718 2.437 2.536 2.525 2.559 EL Std Dev .558 .573 .733 .676 .642 .642 N 108 110 108 110 110 110 Mean 2.466 2.717 2.366 2.415 2.413 2.421 EM Std Dev .508 .560 .678 .664 .648 .608 N 311 311 311 311 311 311 Mean 2.772 2.994 2.762 2.616 2.640 2.848 ELA Std Dev .633 .637 .742 .832 .727 .680 N 84 84 84 84 84 84 Mean 2.801 2.923 2.791 2.777 2.770 2.743 MTH Std Dev .626 .686 .731 .750 .713 .741 N 74 74 74 74 74 74 Mean 2.679 2.947 2.622 2.638 2.596 2.590 HSS Std Dev .551 .538 .747 .678 .653 .643 N 94 94 94 94 94 94 Mean 2.845 3.081 2.735 2.727 2.848 2.833 SCI Std Dev .467 .526 .570 .652 .595 .521 N 66 66 66 66 66 66 Mean 2.854 3.146 2.688 2.656 2.781 3.000 WL Std Dev .755 .798 .854 .811 .752 .856 N 16 16 16 16 16 16 Mean 3.259 3.519 3.056 3.000 3.278 3.444 ART Std Dev .687 .747 .846 .935 .712 .527 N 9 9 9 9 9 9 Mean 2.900 3.000 2.667 2.833 2.833 3.167 MUS Std Dev 1.040 .882 1.443 .289 1.607 1.041 N 3 3 3 3 3 3 Mean 2.433 2.667 2.500 2.000 3.000 2.000 PE Std Dev ...... N 1 1 1 1 1 1 Mean 2.623 2.846 2.540 2.556 2.568 2.601 ALL Std Dev .575 .606 .726 .711 .684 .666 SUBJECTS N 766 768 766 768 768 768 Notes: (a) Sample sizes represent total number of Teaching Events (with the average score of double scored TEs and Calibration TEs included). Total number of scored Teaching Events was 769. (b) Differences in Total MIS are significant across subject areas. Post-hoc comparisons for Total MIS across subject areas are in Table 2.
EL – Multiple Subject Literacy SCI – Single Subject Science EM – Multiple Subject Mathematics WL – Single Subject World Languages ELA – Single Subject English Language Arts ART – Single Subject Art MTH – Single Subject Mathematics MUS – Single Subject Music
Source: PACT Central © 2008 PACT Last edited: 2006 Page 1 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
HSS – Single Subject History-Social Science PE – Single Subject Physical Education Table 2. Post-hoc Comparisons of Total Mean Item Scores across Subject Areas (I) Subject (J) Subject Mean Std. Error Sig. Area Area Difference (I-J) Total Mean Item Score EL EM .091 .061 .985 ELA -.215 .088 .346 MTH -.244 .090 .197 HSS -.122 .078 .973 SCI -.288* .079 .010 WL -.297 .196 .989 ART -.702 .235 .353 EM ELA -.306* .075 .002 MTH -.335* .078 .001 HSS -.212* .064 .030 SCI -.379* .064 .000 WL -.388 .191 .820 ART -.793 .231 .211 ELA MTH -.029 .100 1.000 HSS .093 .089 1.000 SCI -.073 .090 1.000 WL -.082 .201 1.000 ART -.487 .239 .870 MTH HSS .122 .092 .997 SCI -.044 .093 1.000 WL -.053 .202 1.000 ART -.458 .240 .920 HSS SCI -.166 .081 .695 WL -.175 .197 1.000 ART -.581 .236 .642 SCI WL -.009 .197 1.000 ART -.414 .236 .965 WL ART -.405 .297 997 The mean difference is significant at the .05 level. Note: MUS, and PE were excluded because of low N. .
Table 3. Number of Scored Teaching Events by Campus and Subject Area* Campus EL EM ELA MTH HSS SCI WL ART MUS PE Tota l UC Berkeley 0 20 16 0 0 0 0 0 0 0 36 UC Irvine 0 73 17 17 21 9 6 2 3 0 148 UCLA 67 33 0 12 23 18 0 0 0 0 153 UC Riverside 0 67 10 12 10 3 0 0 0 0 102 UC Santa Barbara 0 46 12 5 11 10 4 5 0 0 93 UC Santa Cruz 33 14 12 10 13 9 0 0 0 0 91 Stanford 0 17 16 12 16 14 4 0 0 0 79 San Diego City 10 0 0 6 0 3 0 0 0 0 19 Schools Sacramento State 0 0 1 0 0 0 2 2 0 1 6 San Francisco State 0 41 0 0 0 0 0 0 0 0 41 TOTAL 110 311 84 74 94 66 16 9 3 1 36 * Based on scores submitted by October 12, 2006
Source: PACT Central © 2008 PACT Last edited: 2006 Page 2 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
Table 4. Rubric Scores (All Subjects) 2004-05 Pilot Year Guiding Question N Mean Std Dev Planning 1: Establishing a balanced instructional focus 769 2.939 769 Planning 2: Making content accessible 769 2.836 769 Planning 3: Designing assessments 769 2.763 769 Instruction 1: Engaging students in learning 766 2.593 766 Instruction 2: Monitoring student learning during instruction 767 2.485 767 Assessment 1: Analyzing student work from an assessment 769 2.676 769 Assessment 2: Using assessment to inform teaching 769 2.435 769 Reflection 1: Monitoring student progress 768 2.545 768 Reflection 2: Reflecting on learning 769 2.590 769 Academic Language 1: Understanding language demands 769 2.597 769 Academic Language 2: Supporting academic language 767 2.603 767 development
Source: PACT Central © 2008 PACT Last edited: 2006 Page 3 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
SCORE SUMMARY - TASK MEAN ITEM SCORE FREQUENCIES
Table 5. Total MIS Distribution – All Subjects Table 6. Planning MIS Distribution – All Subjects Score Range Frequency Percent Score Range Frequency Percent 1.00-1.49 6 0.78 1.00-1.49 4 0.52 1.50-1.99 78 10.14 1.50-1.99 17 2.21 2.00-2.49 261 33.94 2.00-2.49 204 26.53 2.50-2.99 209 27.18 2.50-2.99 130 16.91 3.00-3.49 143 18.60 3.00-3.49 297 38.62 3.50-4.00 70 9.10 3.50-4.00 117 15.21 Missing* 2 0.26 Total 769 100 Total 769 100 MIS=Mean Item Score MIS=Mean Item Score * Total MIS could not be computed for TEs with missing scores in whole tasks
Total MIS Distribution Planning MIS Distribution
300 350 250 300 y y 250 c
200 c n n e
e 200 u
150 u q q 150 e e r
100 r F
F 100 50 50 0 0
MIS Range MIS Range
Figure 1. Figure 2.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 4 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
SCORE SUMMARY - TASK MEAN ITEM SCORE FREQUENCIES (Continued)
Table 7. Instruction MIS Distribution – All Table 8. Assessment MIS Distribution – All Subjects Subjects Score Range Frequency Percent Score Range Frequency Percent 1.00-1.49 14 1.82 1.00-1.49 23 2.99 1.50-1.99 65 8.45 1.50-1.99 59 7.67 2.00-2.49 222 28.87 2.00-2.49 233 30.30 2.50-2.99 180 23.41 2.50-2.99 159 20.68 3.00-3.49 159 20.68 3.00-3.49 175 22.76 3.50-4.00 129 16.78 3.50-4.00 118 15.34 Total 769 100.00 Missing 2 0.26 MIS=Mean Item Score Total 769 100 MIS=Mean Item Score
Assessment MIS Distribution Instruction MIS Distribution
250 250 200 200 y c y n
c 150 e
n 150 u e q
u 100 e q 100 r e F r 50 F 50 0 0
MIS Range MIS Range
Figure 4. Figure 3.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 5 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
SCORE SUMMARY - TASK MEAN ITEM SCORE FREQUENCIES (Continued)
Table 9. Reflection MIS Distribution – All Subjects Table 10. Academic Language MIS Distribution – Score Range Frequency Percent All Subjects 1.00-1.49 12 1.56 Score Range Frequency Percent 1.50-1.99 43 5.59 1.00-1.49 7 0.91 2.00-2.49 259 33.68 1.50-1.99 29 3.77 2.50-2.99 158 20.55 2.00-2.49 264 34.33 3.00-3.49 167 21.72 2.50-2.99 158 20.55 3.50-4.00 130 16.91 3.00-3.49 183 23.80 Total 769 100.00 3.50-4.00 128 16.64 MIS=Mean Item Score Total 769 100.00 MIS=Mean Item Score
Reflection MIS Distribution Academ ic Language MIS Distribution
300 300 250 250 y y c 200 c 200 n n e e u 150 u 150 q q e e r 100 r 100 F F 50 50 0 0
MIS Range MIS Range
Figure 5. Figure 6.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 6 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
SCORE SUMMARY– RUBRIC SCORE FREQUENCIES (ALL SUBJECTS)
Planning 1 Score Frequencies Planning 3 Score Frequencies
500 450 450 400 400 350 350 300 y y
300 c c n n 250 e e
u 250 u q q 200 e e r
200 r F F 150 150 100 100 50 50 0 0 1 or 1.5 2 or 2.5 3 or 3.5 4 1 or 1.5 2 or 2.5 3 or 3.5 4 Rubric Score Rubric Scores
Figure 7. Figure 9.
Planning 2 Score Frequencies Ins truction 1 Score Frequencies
400 350 350 300 300 250 y
y 250 c c n n 200 e e
u 200 u q q
e c
e 150 r r
F 150 F 100 100 50 50
0 0 1 or 1.5 2 or 2.5 3 or 3.5 4 1 or 1.5 2 or 2.5 3 or 3.5 4 Rubric Scores Rubric Score s
Figure 8. Figure 10.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 7 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
SCORE SUMMARY– RUBRIC SCORE FREQUENCIES (ALL SUBJECTS)
Ins truction 2 Score Fre que ncies Assessm ent 2 Score Frequencies
400 400 350 350 300 300 s y e i 250 c 250 c n n e e
200 u 200 u q q e r e 150 150 r F F 100 100 50 50 0 0 1 or 1.5 2 or 2.5 3 or 3.5 4 1 or 1.5 2 or 2.5 3 or 3.5 4 Rubric Score s Rubric Scores
Figure 11. Figure 13.
Assessm ent 1 Score Frequencies 2005 Re fle ction 1 Score Frequencies
350 400
300 350
250 300 s y e i
c 250 c
200 n n e e
u 200 u 150 q q e r e 150 r F
F 100 100 50 50 0 0 1 2 3 4 1 or 1.5 2 or 2.5 3 or 3.5 4 Rubric Scores Rubric Score s
Figure 12. Figure 14.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 8 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
SCORE SUMMARY– RUBRIC SCORE FREQUENCIES (ALL SUBJECTS)
Reflection 2 Score Frequencies Acade m ic Language 2 Score Freque ncies
400 450 350 400 350 300 y
y 300 c 250 c n n 250 e e u u 200 q q 200 e e r r 150 F
F 150 100 100 50 50 0 0 1 or 1.5 2 or 2.5 3 or 3.5 4 1 or 1.5 2 or 2.5 3 or 3.5 4 Rubric Scores Rubric Score s
Figure 15. Figure 17.
Academ ic Language 1 Score Frequencies
400
350 300 y c 250 n e
u 200 q e r 150 F
100 50
0 1 or 1.5 2 or 2.5 3 or 3.5 4 Rubric Scores
Figure 16.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 9 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
Trends in Mean Item Scores 2004-2006 (ALL SUBJECTS)
3.50
3.00 s e
r 2.50 o
c Mean Item Scores 2004 S
2.00 Mean Item Scores 2005 m e t
I Mean Item Scores 2006
1.50 n a e 1.00 M
0.50
0.00
Figure 18.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 10 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
Trends in EL Mean Item Scores 2004-2006
3
2.5 s e r
o 2 c EL Mean Item Scores 2004 S
m 1.5 EL Mean Item Scores 2005 e t I
EL Mean Item Scores 2006 n 1 a e M 0.5
0
Figure 19.
Trends in EM Mean Item Scores 2004-2006
3
2.5 s e r
o 2 c EM Mean Item Scores 2004 S
m 1.5 EM Mean Item Scores 2005 e t
I EM Mean Item Scores 2006
n 1 a e
M 0.5
0
s Figure 20.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 11 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
Trends in ELA Mean Item Scores 2004-2006
3.5
3
s 2.5 e r o
c ELA Mean Item Scores 2004
S 2
m ELA Mean Item Scores 2005 e t
I 1.5 ELA Mean Item Scores 2006 n a e 1 M
0.5
0
Figure 21.
Trends in MTH Mean Item Scores 2004-2006
3.5
3 s
e 2.5 r o
c MTH Mean Item Scores 2004
S 2
m MTH Mean Item Scores 2005 e t
I 1.5 MTH Mean Item Scores 2006 n a
e 1 M 0.5
0
Figure 22.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 12 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
Trends in HSS Mean Item Scores 2004-2006
3.5
3
2.5 s e r o c 2 HSS Mean Item Scores 2004 S
m HSS Mean Item Scores 2005 e t I
1.5 HSS Mean Item Scores 2006 n a e
M 1
0.5
0
Figure 23.
Trends in SCI Mean Item Scores 2004-2006
3.5
3
2.5 s e r o c 2 SCI Mean Item Scores 2004 S
m SCI Mean Item Scores 2005 e t I 1.5 SCI Mean Item Scores 2006 n a e
M 1
0.5
0
Source: PACT Central © 2008 PACT Last edited: 2006 Page 13 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
Figure 24.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 14 of 14 Performance Assessment for California Teachers – Pilot Year 4 (2005-06)
Trends in WL Mean Item Scores 2004-2006
4 3.5
s 3 e r o
c 2.5 WL Mean Item Scores 2004 S
m 2 WL Mean Item Scores 2005 e t I 1.5 WL Mean Item Scores 2006 n a e 1 M 0.5 0
Figure 25.
Source: PACT Central © 2008 PACT Last edited: 2006 Page 15 of 14