End of Year Results 2014
Total Page:16
File Type:pdf, Size:1020Kb
End of Year Results 2014 Supported by The David & Elaine Potter Foundation The Isibindi Trust Numeric 2014 Full Year Results Cape Town, January 15, 2015 Numeric provides afterschool maths programs and teacher training programs that use technology-based resources to create exciting and high impact learning environments. Highlights for the 2014 year: Key Metrics 2013 2014 Change Partner schools 10 26 160% Learners tested 2065 4610 123% Applications received 596 1822 206% Application rate 29% 40% 11% Learners accepted 256 1019 298% Acceptance rate 43% 56% 13% Afterschool classrooms 21 46 119% Persistence rate 66% 74% 8% Gross shift in test scores 7.5% 12.4% 4.9% Net shift in test scores 5.1% 7.8% 2.7% Delta 0.34 0.60 0.26 Performance Metrics • The overall persistence rate was 74% compared with 66% in 2013. Persistence is the percentage of learners who remain on-program for the full year. • The average Numeric learner scored 46.1% at endline compared with 33.7% at baseline, a gross shift of 12.4% (2013: 7.5%) • The average non-Numeric learner scored 32.1% at endline compared with 27.5% at baseline, a gross shift of 4.6% (2013: 2.4%) • The net shift attributable to Numeric was 7.8% compared with 5.1% in 2013 • The delta attributable to Numeric was 0.60 compared with 0.34 in 2013. The delta is a statistical measure of impact and Numeric targets a range of 0.5 – 1.0. • Between baseline and endline tests: - 73% of Numeric learners improved by more than 5% (non-participants 46%) - 52% of Numeric learners improved by more than 10% (non-participants 24%) - 34% of Numeric learners improved by more than 15% (non-participants 12%) - 23% of Numeric learners improved by more than 20% (non-participants 5%) - 13% of Numeric learners improved by more than 25% (non-participants 3%) - 6% of Numeric learners improved by more than 30% (non-participants 1%) • Numeric learners accounted for 67 of the 100 most improved learners out of a total 4610 learners tested. Numeric learners account for 23% of all learners tested. Commentary The increased focus on coach training, the implementation of regular testing, the investment of time upfront in finding the right partner schools, and improved learner management systems all contributed to the strong set of results in 2014. The two new clusters in Johannesburg made a positive contribution to these results, generating above average delta and persistence rates. We are similarly pleased with the improved operating performance in Cape Town, with particularly good results achieved in the Mfuleni and Mitchells Plain clusters. The opening of an office in Johannesburg (10 new schools) and the expansion of the Cape Town programs (6 new schools) brought the total number of Numeric partner schools to 26 for the year. It is pleasing to see that in spite of the rapid growth, the program managers and coaches succeeded in improving the quality and impact of our programs. 2 2014$Improvement$Chart$ 60%# 52%$ Numeric$ 50%# NonAParCcipants$ 40%# 34%$ 30%# 24%$ 23%$ %$of$Learners$ 20%# 12%$ 13%$ 10%# 5%$ 6%$ 3%$ 1%$ 0%# >10# >15# >20# >25# >30# %$Improvement$from$Baseline$to$Endline$ 2013$Improvement$Chart$ 60%# Numeric$ 50%# NonBParDcipants$ 40%# 35%$ 30%# 22%$ 22%$ %$of$Learners$ 20%# 12%$ 11%$ 10%# 6%$ 7%$ 3%$ 4%$ 1%$ 0%# >10# >15# >20# >25# >30# %$Improvement$from$Baseline$to$Endline$ 3 Explanation of Results Tables In reading the results tables in this report, the following terminology should be noted: 1. F: Full year students. These are students who appear on Numeric registers in Term 1 and Term 4, in other words, they attend throughout the year. 2. D: Dropped students. These are students who appear on Numeric registers in Term 1 but do not appear in Term 4, in other words they drop out during the year. 3. F&D: Both full year and dropped students, i.e. all students who appear on Numeric registers in Term 1. 4. NP: Non-participating students. These are learners who never enrolled in Numeric’s program to begin with. 5. Persistence Rate: Percentage of students who attend throughout the year and do not drop out. This is computed as F/(F+D). 6. SD: Standard deviation. The standard deviation is a measure of the dispersion of test scores. On a normal curve, 68% of learners fall within 1 s.d. of the mean. That is to say, if the mean test score is 40% and the standard deviation is 10%, then 68% of learners are scoring between 30-50%. 7. Gross shift (%): Average endline score minus average baseline score for a given cohort of learners. 8. Net shift (%): The net shift in test scores attributable to Numeric. This is computed as: (Gross shift for Numeric learners) - (Gross shift for non-participating learners) 9. Delta: The delta measures the improvement in test scores attributable to Numeric in terms of standard deviations from the mean. It is computed as (Net shift) / (Standard deviation). For example, if the net shift for Numeric learners was 5% and the standard deviation on the test was 10%, the delta is 0.5. All results in this report are identical to those in the endline evaluation report provided by Numeric’s independent assessment committee (IAC), with the exception of the delta computations. The IAC report used baseline standard deviations to compute delta. The baseline standard deviation was smaller than the endline standard deviation across the board, and as such, the IAC report overstates the delta. In this report, all deltas have been restated using the average of the baseline and endline standard deviation. Explanation of the Delta The delta is the most important performance metric used to assess Numeric’s operations in any given year. In statistics, the delta (or effect size) is a quantitative measure of the strength of a phenomenon. It is defined as the mean difference between the treatment and control group divided by the standard deviation. �1 − �2 ����� = � In the case of Numeric, �1 is the difference between baseline and endline scores for Numeric learners, while �2 is the difference between baseline and endline scores for non-Numeric learners. The standard deviation (or �) is the average standard deviation for both groups at baseline and endline. Jacob Cohen - an American statistician and psychologist best known for his work on statistical power and effect size - defined effect sizes as "small, d = .2," "medium, d = .5," and "large, d =.8". For a more detailed discussion on the delta and its interpretation, please refer to Appendix D. 4 F&D$ F&D$ F&D$ NP$ NP$ NP$ Net$Shift SD Delta Baseline Endline$ Mark Baseline Endline Mark (%) (Average) Table 1. Mark$(%) Mark$(%) Change Mark$(%) Mark$(%) Change Bardale'Primary 30.8 43.3 12.5 24.6 27.5 2.9 9.6 13.2 0.7 Bardale'Secondary 26.9 39.8 12.9 23.4 26.4 3.0 9.9 11.3 0.9 Results by Manzomthombo'Senior'Secondary 25.1 38.5 13.4 20.8 23.4 2.6 10.8 10.3 1.1 Mfuleni'High 28.0 45.1 17.1 22.5 29.5 7.0 10.1 11.1 0.9 Mfuleni$Schools 27.7 41.7 14.0 22.8 26.7 3.9 10.1 11.5 0.89 Eisleben'Primary 49.8 58.5 8.7 41.1 45.4 4.3 4.4 17.2 0.3 Cluster Imperial'Primary 52.4 65.1 12.7 44.0 50.1 6.2 6.5 18.2 0.4 Lentegeur'High 30.0 41.4 11.4 26.2 30.8 4.6 6.8 11.8 0.6 Oval'North'High 39.0 48.4 9.5 32.6 36.8 4.1 5.3 14.7 0.4 Spine'Road'High 55.1 70.9 15.7 53.6 67.6 14.1 1.7 13.6 0.1 Mitchells$Plain$Schools 45.3 56.9 11.6 39.5 46.1 6.7 4.9 15.1 0.34 Belgravia'High 41.7 49.3 7.7 39.4 41.9 2.5 5.1 15.5 0.3 Bridgetown'High 31.4 32.3 0.9 28.0 29.2 1.2 N0.4 13.4 0.0 Garlandale'High 38.6 44.9 6.3 32.3 35.4 3.1 3.2 12.9 0.2 Ned'Doman'High 27.9 31.9 4.0 25.9 21.4 N4.5 8.5 11.8 0.7 Spes'Bona'High 31.6 42.1 10.5 28.3 34.6 6.3 4.2 14.6 0.3 Sunnyside'Primary 43.8 56.9 13.2 38.5 46.2 7.6 5.5 16.9 0.3 Athlone$Schools 37.4 45.1 7.7 33.3 37.5 4.2 3.5 14.7 0.23 Cape$Town$Average 37.4 48.3 10.9 32.5 37.5 5.0 5.9 13.9 0.46 Ikaneng'Primary 28.0 35.4 7.4 21.5 24.0 2.5 4.9 10.3 0.5 Job'Rathebe'Secondary 26.8 38.6 11.9 20.4 23.1 2.7 9.2 11.0 0.8 Lulama'Primary 38.3 55.5 17.1 22.9 27.3 4.4 12.7 16.5 0.8 Thaba'Jabula'Secondary 31.8 44.8 13.0 22.0 26.1 4.1 8.9 12.2 0.7 Vulamazibuko'Combined'(Gr'7) 31.4 46.2 14.8 27.2 33.8 6.6 8.1 13.2 0.6 Vulamazibuko'Combined'(Gr'8) 30.8 60.1 29.3 22.6 36.9 14.3 15.0 13.4 1.1 Soweto$(Diepkloof)$Schools 31.2 46.8 15.6 22.8 28.5 5.8 9.8 12.8 0.76 Bhukulani'High 27.7 39.0 11.3 23.4 30.2 6.8 4.5 11.3 0.4 Daliwonga'High 27.3 46.5 19.2 23.0 27.4 4.4 14.8 12.3 1.2 Emshukantambo'High 33.2 45.2 12.0 23.8 27.4 3.5 8.4 11.7 0.7 Tiakeni'Primary'(Grade'6) 23.4 38.7 15.4 15.4 20.2 4.8 10.6 12.0 0.9 Tiakeni'Primary'(Grade'7) 29.4 42.7 13.4 17.7 21.9 4.2 9.1 15.1 0.6 Vukazenzele'Primary 29.1 41.8 12.7 20.4 20.6 0.3 12.4 14.1 0.9 5 Soweto$(Mofolo$North)$Schools 28.3 42.3 14.0 20.6 24.6 4.0 10.0 12.8 0.78 Johannesburg$Average 29.8 44.5 14.8 21.7 26.6 4.9 9.9 12.8 0.77 Global$Average 33.9 46.6 12.7 27.5 32.5 4.9 7.7 13.4 0.60 Results Commentary Refer to the table on the prior page for the full set of results by cluster.