USAID / Aprender a Ler (ApaL)
Contract No. AID-656-C-12-00001
FY 2013 1st Quarterly Progress Report: October-December 2012
1. Project Duration: 3 years with additional option year USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Date of Submission: February 7, 2013 Page | 1
Acronyms & Key Terms
ApaL Aprender a Ler (Learn to Read) AWP Annual Work Plan CLIN Contract Line Item Number IEG Impact Evaluation Group MINEC Ministry of Foreign Affairs MINED Ministry of Education DPEC MINED Province Level (N-Nampula, Z-Zambézia) SDEJT MINED District level UTSA University of Texas - San Antonio USAID United States Agency for International Development WEI World Education Inc. IFP Teacher Training Institute
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 2
Table of Contents
1. General Project Information ...... 4 1.1 Project Duration: 3 years with additional option year ...... 4 1.2 Start Date: July 18, 2012 ...... 4 1.3 Life of project funding: $16,200,000.00 ...... 4 1.4 Geographic Focus: Nampula and Zambézia provinces ...... 4 2. Project Objectives ...... 4 Key expected results ...... 4 3. Summary of the reporting period ...... 5 3.1 Period Highlights: ...... 5 3.2 Key Challenges this Quarter: ...... 5 4. Project Performance Indicators ...... 6 5. Major Implementation Issues ...... 18 6. Collaboration with other donor projects ...... 19 7. Key Activities Planned for Next Quarter ...... 19 8. Evaluation/ Assessment Update ...... 22 9. Administrative Update: ...... 22 10. Success Stories and photos: ...... 23 11. Financial Information: ...... 25
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 3
USAID/Aprender a Ler (ApaL) Project FY 2013 1st Quarterly Progress Report: October-December 2012
1. General Project Information
1.1 Project Duration: 3 years with additional option year
1.2 Start Date: July 18, 2012
1.3 Life of project funding: $16,200,000.00
1.4 Geographic Focus: Nampula and Zambézia provinces
2. Project Objectives
The overall goal of the USAID/Aprender a Ler (ApaL) – Learn to Read – project is to improve reading outcomes of students in grades 2 and 3 in over 1,000 urban and rural target schools in the Zambézia and Nampula provinces.
The project will be implemented using participatory and learning-by-doing approaches, based on the many years of WEI experience, to strengthen the technical and organizational skills of a wide range of local counterparts in line with established structures and processes to ensure the sustainability of the project. An ultimate measure of the success is that the institutions at the provincial and district education levels will be able to continue the project activities by the 2015 school year.
Thus, the project is aligned with the efforts of the Mozambican authorities to improve quality of education as stated in the Strategy Plan for Education 2012-2016.
Key expected results Improved quality and increased quantity of reading instruction in grades 2-3 in target schools Improved in-service teacher training and coaching in reading instruction Increased availability of learning materials to students and teachers in grades 2-3 Improved in-service school director training and strengthened school management
To achieve these goals, USAID/ApaL will: 1) train over 5,000 teachers in early grade reading instruction and continuous learning assessment in reading and over 1,000 school directors in school management to increase classroom reading time; 2) develop high-quality reading and instructional materials for students and teachers; and 3) develop and adapt summative and formative assessment instruments to measure improvements in reading ability, instruction quality, and school management skills.
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 4
3. Summary of the reporting period
3.1 Period Highlights: Active involvement of local education institution personnel in the development of ApaL project materials and instruments, including DPEC Planning Department personnel involvement in EGRA and SMA pilot testing (training participation and data collection), and close involvement of IFP teacher trainers in development of teacher and school director training modules. During this quarter, ApaL completed the timely adaptation, pilot testing, and final submission of the Early Grade Reading Assessment and School Management Assessment instruments to the Impact Evaluation Contractor for review. In October the ApaL team and UTSA representatives held Workshops #1 and #2 of the book development series for the first Cohort of Teaching and Learning Aids. In this workshop, local teachers, teacher trainers, and education officials learned how to write stories appropriate for use in teaching young children to read. The workshop resulted in a total of 27 draft stories which will later be refined by ApaL and UTSA reading experts as books distributed to schools in the 2013 school year. This workshop was an exciting process for the participants (see success stories section below) and drew positive attention to ApaL activities in the provinces. In November, South African illustration expert, Vusi Malindi, an associate of the UTSA team, traveled to Nampula to hold a capacity building workshop with local illustrators. This workshop strengthened their skills in creating illustrations that help explain a text and further children’s ability to understand it. ApaL plans to move forward with illustration contracts for project books in the next quarter. A task analysis exercise was conducted with school directors and teachers to inform development of training materials and assessment instruments. This participatory process also helped introduce ApaL to school directors and begin a process of self-reflection regarding school management practices. All project expected staff were hired in all provinces and are working 100% time on their areas of responsibilities. The Local Education Institution Capacity Building position has been replaced and additionally the project hired the M&E National Coordinator, both of whom started working in December. During this quarter the process of registration was finalized and WEI was issued an approval to operate in the following provinces: Maputo, Nampula and Quelimane in Mozambique. Due to the many administrative and bureaucratic requirements for this, registration is usually a lengthy process. ApaL’s proactive approach which achieved a rapid completion of the registration process is considered a prominent success. ApaL’s receipt of authorization from MINEC supported a smooth registration process in Nampula and Zambézia provinces, despite initial resistance from local entities.
3.2 Key Challenges this Quarter: The replacement of the Local Education Institutional Capacity Building Specialist caused delays in implementation of institutional gap analysis activities. The postponed activities have been planned for the following quarter and will be supported by WEI experts in Boston to ensure timely completion of the IGA instruments per Year One AWP requirements. Contractual and administrative delays at UTSA have resulted in delays in the book development process. Delivery of draft and final versions of Cohort 1 books was repeatedly delayed by the
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 5
UTSA team, causing further delays in the finalization of the books and the development of associated Teachers Guides. ApaL has experienced significant delays in work completed by international subcontractor, UTSA, in addition to time-consuming administrative and contractual disputes. Due to UTSA’s unwillingness to agree to contract terms regarding copyrights for ApaL materials and exclusive branding requirements (included in the subcontract per requirements contained in USAID Direct Contracting Procedures and ApaL’s approved Branding Strategy and Marking Plan), a full subcontract has yet to be executed with UTSA. Work progressed during this quarter under a Letter of Authorization mechanism as negotiations continued, however, these contractual issues have impeded the materials development process and UTSA has delayed delivery of materials and completion of work. In particular, contractual and administrative delays at UTSA have resulted in delays in the book development process. Delivery of draft and final versions of Cohort 1 books was repeatedly delayed by the UTSA team, causing further delays in the finalization of the books and the development of associated Teachers Guides. The need for translation of the stories from Portuguese to English and from English back to Portuguese and UTSA team members’ lack of understanding of the Mozambican context resulted in delivery of books which required further revision by the local ApaL team, which caused further delays in the materials development process.
3.3 Summary of STTA During this Quarter: WEI Senior Program Officer, Corrie Blankenbeckler, traveled to Mozambique October 7-17, 2012 to support the ApaL team in drafting the Mobilization Period Report, support Annual Work Plan and budget revision processes, and work with the team to plan the development of school management instruments and materials. WEI Senior Monitoring and Evaluation Advisor, Dr. Valeria Rocha, traveled to Mozambique November 3-20, 2012 to begin a development of the Performance Management Plan (PMP) with the ApaL team, assist in the selection of the National Monitoring and Evaluation Coordinator, and support the finalization of the EGRA and SMA instruments. International subcontractor UTSA’s materials development team (Dr. Misty Sailors, Dr. Jim Hoffman, and Troy Wilson) traveled to Mozambique October 4-17, 2012 to work with the ApaL team in facilitating book development workshops #1 and #2 and begin planning for school management materials development.
4. Project Performance Indicators During this reporting period, project activities are reported based on the Year One Annual Work Plan, as the Performance Management Plan (PMP) is pending USAID approval. During the next quarter, ApaL project monitoring and evaluation systems and processed will be established to allow reporting based on the PMP following final approval.
Overall Program Management and Capacity Building Task A: Engage Stakeholders and Build Institutional Capacity in Reading Instruction and Management Key Successes and Successes: Challenges Active involvement of local education institution personnel in the development of ApaL project materials and instruments, including DPEC Planning Department personnel involvement in EGRA and SMA pilot testing (training participation and data collection), and close USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 6
involvement of IFP teacher trainers in development of teacher and school director training modules. Apal was presented to the Governor of Zambézia during a meeting in his office who ensured any support necessary.
Challenges: The replacement of the Province Director of Education in Zambézia meant the mobilization process of the DPEC had to start over again including formal cooperation agreements as well as setting up the ApaL Provincial Coordinator Group. The replacement of the Local Education Institutional Capacity Building Specialist caused delays in implementation of institutional gap analysis activities. The postponed activities have been planned for the following quarter and will be supported by WEI experts in Boston to ensure timely completion of the IGA instruments per Year One AWP requirements. Although necessary for the sustainability of project interventions, ensuring active involvement of local government representatives has also resulted in a slower process of materials development. A.1: Orient and Collaboration with MINED at national, provincial, and district levels: During engage local this quarter, ApaL staff held a series of consultative meetings with MINED education personnel at national, provincial, and district levels to facilitate active institutions and involvement in the project. ApaL initially planned to draft and sign personnel in ApaL Memoranda of Understanding (MOUs) with each of the relevant institutions. However, since a contractual agreement already exists between USAID and MINED it was agreed that there was no need for ApaL to draft another such document. Instead, ApaL staff worked this quarter to begin drafting sample Letters of Commitment (LOCs) for each of the relevant institutions (, DPECs, IFPs,). ApaL plans to finalize and sign these LOCs with the relevant institutions in the next quarter.
Target District Selection: Target districts for Year One interventions were selected based primarily on accessibility criteria—districts in Nampula and Zambézia provinces located along the main corridor between Nampula and Qualimane cities were selected. In addition, it was necessary to select schools not receiving any other interventions or donor-funded project interaction and those which are not part of MINED’s pilot bilingual education program, as this program would conflict with the interventions contained in ApaL. The final district as selected and sent to the IE for ZIP selection and random assignment into full, medium treatment, or control groups were as follows:
Nampula Province: Nampula City Monapo Murrupula Nampula District (Rapale)
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 7
Zambézia Province: Quelimane City Mocuba Nicoadala
Within each district, the IE selected ZIPs based on their size (looking to include ZIPs of similar size as much as possible to each intervention implementation and assessment activities). 15 ZIPs in Nampula province were selected and then divided into three groups with approximately equal numbers of schools which were then randomly assigned to full treatment, medium treatment, and control assessment groups. In Zambézia 15 ZIPs were also selected and randomly assigned to assessment groups following the same process. A.3: Establish ApaL has focused this quarter on the active involvement of local education Taskforce and institutions in the development of project materials to ensure full local Steering ownership and understanding of the materials developed and Committees contextualization of all materials to the realities faced by teachers and school directors in ApaL schools. This quarter ApaL entered into discussions with provincial authorities and has received support for the establishment of provincial-level steering committees. Draft scopes of work have been jointly- developed and further progress is planned for the next quarter. A.4: Conduct joint During this quarter ApaL prioritized developing collaborative relationships institutional gap with local education institutions, in particular DPECs and IFPs to ensure their analysis (IGA) of active participation in materials and instrument design support for ApaL local education project interventions. Given this, initial rapid capacity assessments were institutions and postponed to Quarter 2. personnel and develop capacity building work plan A.5: Implement This quarter, Apal has focused on its learning by doing approach rather than institutional formal training activities. Although more time consuming, this has proven to capacity building be the most effective training approach due to its hands on and results- training program based, concrete activities. Due to the replacement of the Local Education for local education Institution Capacity Building Specialist position in December, the institutions and implementation of the capacity building training program has been delayed. carry out regular Implementation of the institutional gap analysis is planned for next quarter ROAs and capacity building training programs will be developed following initial assessments. While specific capacity building training has been delayed, ApaL has ensured the involvement of local education institution personnel in all project activities and materials development to commence learning-by-doing capacity building as early in project implementation as possible. Task B: Aprender a Ler Project Management Key Successes and Successes: Challenges Full integration of Politécnica into the ApaL project team in each province, including full involvement in materials development, instrument testing, and training activities. Weekly project planning procedures established within the ApaL USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 8
team
Challenges: Establishing efficient communication between ApaL Mozambique team and international subcontractor University of Texas at San Antonio (UTSA) has proven difficult due to administrative and contractual delays at UTSA and UTSA team members’ lack of Portuguese language skills and knowledge of the Mozambican context. B.1: Conduct joint An initial joint capacity mapping activity was conducted with Politécnica staff capacity mapping during the previous period. During this quarter, ApaL has signed a full with A Politécnica subcontract with Politécnica and Politécnica has engaged staff members to determine tasked with carrying out ApaL project activities. Politécnica team members capacity strengths have been fully involved in all materials, instrument, and training and gaps development processes, all training and pilot testing activities, and all engagement of local education authorities. As planned, Politécnica and WEI project staff work together as a united team in each province to implement activities. Due to the replacement of the Local Education Institution Capacity Building Specialist position this quarter, capacity assessment and reinforcement activities have postponed to next quarter. Monthly ApaL project staff and partners meet on a regular basis to conduct joint Management planning exercises. Given the intensive implementation timeline, weekly Meetings plans are developed for each major project component and reviewed regularly with provincial and Maputo-based staff. Task C: USAID Reporting Key Successes and Successes: Challenges Timely submission of Mobilization Period Progress and Financial Reports. Timely submission of Performance Management Plan C.2: Develop Mobilization period progress report submitted to USAID October 31, 2012. Quarterly Progress Reports C.3: Develop Mobilization period financial report submitted to USAID October 31, 2012. Quarterly Financial Reports C.4: Performance PMP Development: In November, 2012, Senior Monitoring and Evaluation Management Plan Advisor, Dr. Valeria Rocha traveled to Mozambique to work with the ApaL team in the development of the Performance Management Plan (PMP). Following discussions with USAID representatives in Maputo and clarification of the PMP template and requirements, the ApaL team gathered in Nampula to review and identify all program components and activities required to achieve project short- and long-term goals. Following this process key technical staff and coordinators developed an instrument to outline project activities, outputs, performance indicators, means of verification, purpose, and data collection responsibilities, which was used as the basis for PMP development. In December, the National Monitoring and Evaluation
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 9
Coordinator, Meldina Valente, was hired and worked closely with Dr. Rocha and the ApaL team to complete and refine the PMP, which was submitted to USAID December 28, 2012. Intermediate Result 1: Improved Quality of Reading Instruction in Grades 2-3 in Target Schools Sub IR 1.1: Improved In-Service Teacher Training and Coaching in Reading Instruction Key Successes and Successes: Challenges Timely adaptation, pilot testing, and final submission of the Early Grade Reading Assessment instrument to the Impact Evaluation Contractor for review.
Challenges: Because pilot testing of the EGRA instrument took place during the school holidays, accessing sufficient numbers of students, teachers, and school directors to test proved difficult and thus limited the scope of the pilot testing process. 1.1.1 Design, Pilot Development/adaptation of EGRA Instrument: Following discussions with and implement the Impact Evaluation contractor, International Business and Technical EGRA instrument Consultants, Inc., and USAID, it was decided that ApaL will use the existing Portuguese EGRA instrument adapted for Mozambique by USAID EQUIP 2 and Aga Khan and used with a sample of students in Cabo Delgado province. Student assessment and reading experts at UTSA selected portions of this instrument and made recommendations for revisions to adapt to ApaL assessment needs. The ApaL local team worked closely with Politécnica to further revise and refine the instrument based on implementation realities in the provinces.
The EGRA instrument assesses the following skills: 1. Listening comprehension (oral vocabulary): ability to understand basic words in Portuguese 2. Concepts about printed materials: ability to understand how the world of print works 3. Letter naming fluency: ability to read the letters of the alphabet fluently 4. Word fluency: ability to read words fluently 5. Reading comprehension: ability to comprehend reading passages associated with timed reading assignments
EGRA Instrument Pilot Testing: On November 16-17, 2012, ApaL held training for EGRA and SMA pilot test enumerators in Nampula. In addition to an overview of EGRA and SMA concepts and objectives and details in instrument implementation and assessment procedures, training focused on providing enumerators with an understanding of the ethical implications associated with working with young children. Training participants included four members of the DPEC Planning Department and 3 students from the Pedagogical University. DPEC involvement was considered crucial even at this early pilot stage to begin capacity building processes and ensure local ownership and buy-in for the instruments developed. Training was facilitated
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 10
by Amilcar Sueia of Aga Khan, Cabo Delgado, selected for his extensive experience with the EGRA and SMA instruments adapted under EQUIP 2 on which ApaL instruments are based.
Following training both instruments were pilot tested in eight schools (seven public, one private) in Nampula city on November 19-23, 2012. A total of 90 students were tested (46 in Grade 2 and 44 in Grade 3). Interviews lasted an average of 40 minutes during the first two days and 25 during the last three days of testing. In addition to instrument adjustments which arose (particularly aimed at increasing students’ comprehension of the questions), the following challenges were noted during the pilot exercise:
1. Some enumerators seemed experienced difficulty completing all the required forms and instruments. This may have been a result of misunderstanding during training or insufficient time allotted for assessment. Simplification of the instruments and forms and more time is needed to ensure quality and accurate data is collected and training must be further clarified to ensure all participants are aware of the data collection requirements. Furthermore, daily supervision during data collection should be strengthened. 2. As the instrument pilot was carried out during school holidays, there were significant difficulties reaching sufficient numbers of children, teachers, and school directors to create a valid pilot sample. During full data collection processes, ApaL should plan carefully to ensure that schools are fully in session and students and teachers are present.
Other recommendations included the following: 1. Including a high proportion of female enumerators who speak the local language (Emakua) is important to ensure that students feel comfortable responding to questions. 2. There is confusion between the EGRA and SMA instruments. During training there must be sufficient separation of the two and clarification of their respective assessment objectives and procedures.
Following the pilot, the instruments were further developed, reviewed, and adapted to the local teaching-learning practices with the ApaL team and Politécnica partners. Final EGRA and SMA instruments were submitted to the IE team as requested in December. 1.1.2 Develop and During this first quarter, ApaL has focused primarily on developing reading implement and instructional materials which will be distributed to school directors, continuous in- teachers, and students. These materials form the backbone of ApaL’s reading service teacher instruction approach and are crucial to the teacher training processes. training/coaching Finalization of the first cohort of these materials has been delayed due to model delivery delays with subcontractor UTSA, because of which the subsequent development of accompanying teacher training materials has also taken place
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 11
later than planned. During this quarter, ApaL and Politécnica reading expert staff have worked closely with local teacher training officials (from IFP Marrere and Nampula) to begin development of the series of teacher training modules and the Teacher’s Manual for Reading Instruction. While close involvement of local education officials has further slowed development, it has also ensured alignment with the curriculum and existing practices of school directors and teachers and helped galvanize local support for ApaL interventions. To date, four modules for continuous in-service training sessions have been developed in draft form and an initial draft of the Teacher’s Manual has been developed. The finalization of the full series of in- service modules and the Teacher’s Manual for a complete set of first generation training materials is planned early in the next quarter. Teacher training (both training of the lead trainers and cascade training at the ZIP level is planned to take place next quarter and continue through Quarter 3. 1.1.3 Develop Development of the evaluation grid has been postponed to the 2nd Quarter, program following further completion of ApaL TLAs, training modules, and other Manual/Materials materials. Evaluation Grid Sub IR 1.2: Increased Availability of Teaching and Learning Materials to Students and Teachers in Grades 2-3 Key Successes and Successes: Challenges In October the ApaL team and UTSA representatives held Workshops #1 and #2 of the book development series for the first Cohort of Teaching and Learning Aids. In this workshop, local teachers, teacher trainers, and education officials learned how to write stories appropriate for use in teaching young children to read. The workshop resulted in a total of 27 draft stories which will later be refined by ApaL and UTSA reading experts as books distributed to schools in the 2013 school year. This workshop was an exciting process for the participants (see success stories section below) and drew positive attention to ApaL activities in the provinces. In November, South African illustration expert, Vusi Malindi, an associate of the UTSA team, traveled to Nampula to hold a capacity building workshop with local illustrators. This workshop strengthened their skills in creating illustrations that help explain a text and further children’s ability to understand it. ApaL plans to move forward with illustration contracts for project books in the next quarter.
Challenges: Contractual and administrative delays at UTSA have resulted in delays in the book development process. Delivery of draft and final versions of Cohort 1 books was repeatedly delayed by the UTSA team, causing further delays in the finalization of the books and the development of associated Teachers Guides. The need for translation of the stories from Portuguese to English and from English back to Portuguese and UTSA team members’ lack of understanding of the Mozambican context resulted in delivery of USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 12
books which required further revision by the local ApaL team, which caused further delays in the materials development process.
1.2.1 Develop set of The participatory development of Teaching and Learning Aids (TLAs), teaching and specifically, a series of Guided Reading Books and Read Aloud Books leveled learning aids (TLAs) for each 2nd and 3rd Grades, has been a primary focus for ApaL during Quarter for use in the 1. The ApaL team worked closely with international subcontractor the classroom University of Texas at San Antonio (UTSA) to begin book development through a series of three workshops, which support Mozambican teachers, teacher trainers, and education officials to author reading books that are both highly relevant to the local context and appropriately leveled for students in Grades 2-3.
Book Authoring and Revising Workshops: In October 2012, the UTSA senior technical team (including Dr. Misty Sailors, Dr. Jim Hoffman, and Mr. Troy Wilson) traveled to Nampula to co-facilitate book development Workshops #1 and #2 with the ApaL team and Politécnica staff. A total of 26 participants from Nampula and Zambézia attended both workshops (including 13 IFP teacher trainers and local education officials and 12 teachers from local schools), which were held at the Nampula Teacher Training Institute. Participants were selected in collaboration with DPECs and IFPs based on their demonstration of interest in children’s books, writing, and commitment to quality education. During Workshop #1: Telling and Gathering Stories (October 8-9, 2012), participants authored and submitted drafts of three stories for consideration. One draft was written from a lived experience. One draft was a story that followed a repetitive pattern. One draft was that of an oral tradition. Authors were encouraged to think about young children as they choose the topic of each story. During Workshop #2: Revising Stories (October 10-12, 2012), the same group of participants continued to revise their stories. The workshop focused on helping authors to tailor their stories to the target audience through the support of mini-lessons covering topics such as content, word choice, and imaging decisions. Both workshops introduced participants to ApaL and provided guidance on project quality criteria and making connections between the books and the existing curriculum. The following 12 “Truths of Writing” formed the outline for the facilitation of the two workshops: Workshop 1: Telling and Gathering Stories Truth 1: Writers need tools Truth 2: Writers write from experience Truth 3: Writers write to an intended audience Truth 4: Writers understand the writing process
Workshop 2: Revising Stories Truth 5: Authors work with other authors to revise their writing Truth 6: Authors sometimes “stretch the truth” to enhance their story Truth 7: Authors work on more than one piece at a time Truth 8: Authors “show” not “tell” readers
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 13
Truth 9: Authors use strong words Truth 10: Authors select words that flow Truth 11: Authors hook their readers Truth 12: Images extend words
A total of 27 stories were developed during the two workshops. Ten of these stories were then selected for the “Urgent 10” books to be given priority attention during the development process.
Book Development Process: Early in Quarter 2, Workshop #3: Field Testing Stories is planned to take place for the first cohort of books. Between Workshops #2 and #3 in November and December the UTSA and Politécnica Development Team began work on the “Urgent 10” books. The development process worked to make the stories more engaging, align them with the national curriculum, and strategically employ an instructional design across the series of books.
Illustrator Capacity Building In November, illustration expert Vusi Malindi, an associate of the UTSA team, traveled to Nampula to hold a capacity building workshop with local illustrators, which will inform selection of illustrators for ApaL Guided Reading Books and Read Aloud Books. This session focused on strengthening their abilities to create quality illustrations which are linked to and help further explain a text. Lessons focused on the ability to read a text in order to create illustrations. Following this ApaL staff worked closely with these illustrators in the development of sample illustrations for the “Urgent 10” titles. 1.2.2 Develop and Teacher Guides Development: Following discussions with local education test Model Lesson authorities, the Model Lesson Plans has been re-termed Teachers Guides. Plans (MLPs) and These Teachers Guides will link one Guided Reading Book and one Read Aloud develop/implement Book together with a series of in-class lessons and activities to take place over Continuous a two week period. A Continuous Classroom Reading Assessment (CCRA) Classroom Reading process is integrated into the Teachers Guide at the end of the two week Assessment (CCRA) teaching period. Because Teachers Guides are developed based on the content of each Guided Reading Book and Read Aloud book, delays at UTSA in the book development process have caused subsequent delays in the development of the Teachers Guides. During this quarter, UTSA and ApaL together developed and refined an instructional model to guide the development of the Teachers Guides. In December the first Cohort of Teachers Guides was delivered by UTSA. The local ApaL team is currently working to contextualize and finalize this first Cohort. An initial training with school directors and pedagogical directors is planned in January which will begin rollout of the training of trainers process. During this training further feedback will be solicited to finalize the materials, which will ensure they are well-suited to the local context and realities faced by local teachers.
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 14
1.2.3 Print, TLA Printing and Distribution Planning: Due to contractual and package and administrative delays within UTSA, book development and delivery of final distribute TLAs, books was delayed. In addition, the ApaL team has found that the “Urgent MLPs, and CCRAs 10” books delivered by UTSA still require further contextualization and revision to ensure they are appropriate for the Mozambique context. Because of this, progress toward a printing and distribution contract has also been delayed. During this quarter several printing and distribution partners were contacted and budget planning for the printing and distribution was begun. Early in Quarter 2, ApaL will complete the selection process for the printing and distribution contract. Intermediate Result 2: Increased Quantity of Reading Instruction in Grades 2-3 in Target Schools SUB IR 2.1: Strengthen School Management Key Successes and Successes: Challenges Timely adaptation, pilot testing, and submission of SMA instrument to IE contractor for review Active involvement of local education institution personnel (specifically IFP Marrere staff) in the development of draft school management training modules Development of draft School Management Rapid Assessment formative instrument Task analysis exercise conducted with school directors and teachers to inform development of training materials and assessment instruments. This participatory process also helped introduce ApaL to school directors and begin a process of self-reflection regarding school management practices.
Challenges: Because the SMA Instrument was pilot tested during the school holidays, enumerators had limited access to schools, school directors, and teachers, which limited the scope of the pilot exercise. While ApaL considers the close involvement of IFP staff and other local education personnel crucial to the sustainability of project interventions, facilitating this involvement throughout the design of training materials slowed the development process slightly. 2.1.1 Design, pilot, School Director Orientation and Task Analysis Execrise: In November, ApaL and implement facilitated a Task Analysis and Responsibility Mapping Exercise with School SMA tool -School Directors (SDs) and Pedagogical Directors (PDs) in Nampula and Zambézia. Management This exercise included the following components: Assessment (SMA) 1) Orientation to ApaL interventions for SDs and PDs in full treatment schools. 2) Information gathering on SD and PD responsibilities to guide development of project assessment instruments, training modules, tools, and materials. 3) A School Management Collective Reflection through which SDs and PDs were able to reflect on their practices with respect to planning, pedagogical and administrative management, organization,
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 15
supervision, evaluation, school governance, and professional development. 4) Introduction to a task analysis instrument, which allows SDs and PDs to collect information on their various areas of responsibility, analyze basic school management competencies, and use this data to improve school management practices. The instrument allows for both short- and long-term data collection and analysis (i.e. weekly, per semester, and annually).
A total of 53 participants were introduced to ApaL interventions, including: 10 SDs, 7 PDs, 4 SD/PDs, and 3 technicians from the District Education Office (SDEJT) from Murrupula and Nampula/Rapale Districts in Nampula; and, 16 SDs, 13 PDs, and 2 SDEJT technicians from Nicoadala and Mocuba Districts in Zambézia.
Participants had the opportunity to collectively analyze the primary challenges facing their schools, with respect to: (i) planning, organization, and management; (ii) teacher motivation; (iii) parent’s participation in their children’s education. The results of this exercise were used in the development of the School Management Rapid Assessment (ApaL formative school management assessment instrument) and school management training modules. While many participants were inexperienced in this type of exercise, it was possible to gather key information necessary for the development of training modules and assessment instruments.
SMA Instrument Development/Adaptation: The SMA instrument will be used to assess school and classroom environment and teachers’ and school directors’ characteristics and practices as they relate to quality and quantity of instruction. Following consultation with USAID, it was decided that ApaL would utilize an existing School Management Assessment instrument in Portuguese already developed and tested in Mozambique by Aga Khan and AED under funding from USAID. Similar to the EGRA instrument, described above, UTSA school management experts selected portions of the existing instrument for inclusion, made recommendations for adaptation, and simplified the instrument (in particular the classroom observation protocol) for use under ApaL. Following this, the ApaL team worked closely with Politécnica to further revise the instrument and prepare for pilot testing (including addition and subtraction of items to ensure the instrument met ApaL assessment needs). The current SMA comprises items from the following domains: 1. Classroom Observation Protocol; 2. School Director Protocol; 3. Teacher Protocol; and, 4. School Observation Protocol.
SMA Enumerator Training and Pilot Testing: Pilot enumerator training took
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 16
place in Nampula on November 16-17, 2012. Enumerators trained included four technicians from the DPEC Department of Planning and three students from Linguistics Department of the Pedagogical University (UP). The SMA instrument was tested on November 19-23, 2012 in eight schools (seven public and one private) in Nampula City and Rapale District. During the pilot, a total of eight school directors and teachers were interviewed. On average, each interview lasted 45 minutes. School directors interviewed expressed interest in the assessment and several directors said that the exercise had helped them think about ways to improve various aspects of their work. In addition to challenges and recommendations listed above in 1.1.1, the pilot also led to changes in the instrument, primarily aimed at improving the ease of comprehension of questions. Develop and Development of School Director Training Modules: During Quarter 1, ApaL implement school- cluster-based worked closely with the Teacher Training Institute of Marrere in the continuous in- development of the first generation of school management training modules. service school These modules are closely linked to ApaL’s reading instruction methodology directors and included information gathered through the task analysis and mapping training/coaching model exercise conducted with directors. Two teacher trainers from IFP Marrere with prior experience working on the Osuewela in-service director training project were actively involved throughout the module drafting process. During this quarter six modules focused on the following topics related to school management were developed and are in the final stages of design: 1. School management formative assessment skills and instruments, and school leadership, planning, and organization; 2. Efficient classroom management practices; 3. Time management to increase academic learning time (ALT) and time on-task; 4. Pedagogical supervision of teachers; 5. Participatory approaches to reading instruction and school management; and, 6. Facilitation/coaching skills for implementation of school management interventions at the ZIP-level.
In addition, modules related to education planning and financial management, inclusive education and coaching practices are in the early stages of development, as is an overall school management training manual. These modules will be tested early in the next quarter with the first SD and PD trainings. Feedback from these sessions will be used to adjust content and finalize the modules. Develop School School Management Rapid Assessment Instrument Development: Using a Management Rapid Assessment (SMRA) framework provided by school management experts at UTSA and information gathered through the task analysis exercise conducted with school managers, the ApaL team worked closely with the Teacher Training Institute of Marrere and Politécnica University in the reformulation and contextualization of four formative assessment protocols which comprise the School Management Rapid Assessment (SMRA) instrument. The SMRA will be used to assess ApaL implementation at the classroom and school levels on a monthly basis.
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 17
Although ApaL initially intended to work with UTSA in the development of the SMRA, their approach proved overly complex for ApaL’s formative assessment needs and for the context of school management in Mozambican schools.
The SMRA instrument includes the following assessment components: 1. Teacher Observation: Conducted by the SD or PD; evaluates a teacher’s knowledge and ability to use methodologies contained in ApaL’s training program. 2. School Management Factors – Conducted by the SD; evaluates the school-level factors, which may influence or impact the implementation of ApaL reading interventions. 3. Teacher Reflection Sheet: Completed by teachers; allows teachers to self-evaluate their reading instruction practices. 4. Student Reflection Sheet: Completed by students; allows students to evaluate the various components of the reading program.
5. Major Implementation Issues
Changes and Key and other senior project personnel: During the Mobilization Period, the Local Education Institution Capacity Building Specialist was dismissed on grounds that he was not performing his scope of work or meeting the quality standards of the ApaL project. During this quarter, following discussions with and subsequent approval by USAID, ApaL hired Ms. Zaida Cabral for this Key Personnel position. While the replacement of this position delayed implementation of initial institutional capacity development activities, WEI is fully confident in her skills, experience, and knowledge of the education sector in Mozambique. In addition, ApaL mitigated the delays in capacity building activities by facilitated active involvement of local education personnel throughout all materials, training, and instrument development and revision processes to ensure learning-by- doing approaches began early in project implementation and to garner their support for ApaL interventions. In addition to Ms. Cabral, ApaL also selected a National Monitoring and Evaluation Coordinator, Ms. Meldina Valente. Ms. Valente came with excellent references and immediately after her hire in December worked closely with WEI Senior Monitoring and Evaluation Advisor Dr. Valeria Rocha on the development of the PMP.
Budget revision: Following submission and USAID approval of the Year One Annual Work Plan, WEI completed a budget revision process to ensure all activities contained in the work plan were accurately reflected in the budget. USAID approval was sought and granted through signing of Modification 02.
Subcontractor delays: ApaL has experienced significant delays in work completed by international subcontractor, UTSA, in addition to time-consuming administrative and contractual disputes. Due to UTSA’s unwillingness to agree to contract terms regarding copyrights for ApaL materials and exclusive branding requirements (included in the subcontract per requirements contained in USAID Direct Contracting Procedures and ApaL’s approved Branding Strategy and Marking Plan), a full subcontract has yet to be executed with UTSA. Work progressed during this
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 18
quarter under a Letter of Authorization mechanism as negotiations continued, however these contractual issues have impeded the materials development process and UTSA has delayed delivery of materials and completion of work.
6. Collaboration with other donor projects
During this quarter, ApaL leveraged the experiences of prior USAID-funded education initiatives in Mozambique. The use of an already-adapted set of EGRA and SMA instruments allowed for more time to be spent contextualizing these instruments to the assessment needs of ApaL and to the context of ApaL schools. ApaL further leveraged this experience by engaging a trainer from Aga Khan who has extensive experience with the existing instruments and excellent training skills. In addition, discussions and coordination with IBIS ensured that ApaL target schools would not overlap with schools where they are implementing interventions, which is crucial to the validity of the Random Control Trial taking place in this first year. ApaL also continues to meet with counterparts from other donor-funded projects and has reached out to contacts at Save the Children in Maputo and Nampula to discuss possibilities for coordination and mutual support.
7. Key Activities Planned for Next Quarter
Program Management and Capacity Building Task A: Engage Stakeholders and Build Institutional Capacity in Reading Instruction and Management A.1: Orient and Program launch engage local Provincial program launches with all stakeholders education institutions and personnel in ApaL
A.3: Establish Establish Reading Taskforce, develop scope of work, and begin Taskforce and quarterly meetings Steering Committees Establish Provincial Steering Group, develop scope of work, and begin quarterly meetings Establish district-level Level Reading Planning Groups and develop scopes of work A.4: Conduct joint Conduct necessary rapid assessments of local education institutions institutional gap (DPEC, IFP, SDEJT, ZIP) analysis of local Develop/adapt IGA instruments and processes education Begin implementation of IGA with local institutions institutions/personnel A.5: Implement Develop initial framework for capacity building with IFPs, DPECs, institutional capacity SDEJTs, ZIPs building training Orientation/institutional capacity building training with IFPs, DPECs, program for local SDEJTs, ZIPs education institutions UGEA quarterly meeting and carry out regular USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 19
ROA Task B: Aprender A Ler Project Management B.1: Conduct joint Develop and begin learning by doing and spot training program with capacity mapping Politécnica with A Politécnica to Continue full and active involvement of Politécnica staff as integrated determine capacity ApaL team members strengths and gaps B.2: Monthly Continue weekly planning processes management Hold monthly ApaL team meetings meetings
Task C: USAID Reporting C.2. Develop Submit Quarter 2 Progress Report on April 30, 2013. Quarterly Progress Reports C.3 Develop Quarterly Submit 2 Financial Report on April 30, 2013 Financial Reports C.3 Performance Negotiate final approved PMP with USAID Management Plan Begin establishing performance monitoring and evaluation procedures, systems, and instruments Intermediate Result 1: Improved Quality of Reading Instruction in Grades 2-3 in Target Schools Sub-IR 1.1: Improved in-service teacher training and coaching in reading instruction 1.1.1 Design, Pilot Review and finalize EGRA and SMA instruments and associated and implement EGRA assessment procedures in collaboration with IE Contractor instruments Conduct EGRA and SMA enumerator training Carry our EGRA and SMA baseline data collection in 180 schools Close monitoring of baseline assessment implementation EGRA in 180 schools ( data collection prior to start intervention) (ref budget 1.1.1d) Monitoring of pre-test implementation (ref budget Politécnica ) 1.1.2 Develop and Provincial level reading instruction and coaching packages presentation implement workshop (ref budget 1.1.2a) continuous in-service Begin radio broadcast development with approved subcontractor teacher TOT workshop for lead trainers (LTs) (IFP TTs to participate as assistant training/coaching facilitators.) (ref budget 1.1.2c) model and TOT of Complete development of first generation teacher training materials pedagogical directors for INSET Saturday sessions (PDs) Closely monitor TOT processes to ensure Lead Trainers are equipped with skills necessary to roll-out training at the ZIP level and conduct additional training, support, or supervision as necessary to ensure quality training of teachers
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 20
1.1.3 Develop Evaluation grid developed program Assessments, and pilot materials reviewed against the checklist and Manual/Materials corrections made Evaluation Grid Equity tools and “icebreaker” activities developed for use in workshops and trainings to raise participant awareness Independent consultant (Mozambican) review of TLAs (for alignment with Mozambican curriculum, language and local context) Sub-IR 1.2: Increased availability of teaching and learning materials to students and teachers in Grades 2-3 in target schools 1.2.2 Develop set of Conduct Workshop #3: Field Testing for Cohort 1 TLAs teaching and learning Carry out second illustrators’ capacity building workshop aids (TLAs) for use in Negotiate and sign contract with local illustrators for Cohort 1 TLA the classroom (and illustrations nd communities in 2 or Complete Cohort 1 TLAs for printing and distribution rd 3 year) Carry out Workshops #1 and #2 for Cohort 2 TLAs Begin development process for Cohort 2 TLAs 1st cohort TLAs testing and finalization (ref budget 1.2.1a) Develop draft proposal for awareness campaigns for parents/children to increase understanding of value of reading at home 1.2.2 Develop and Complete development of Cohort 1 Teacher Guides (including CCRAs) test Model Lesson Solicit feedback through initial training workshops to revise and Plans (MLPs) and complete first generation versions of Cohort 1 Teacher Guides develop/implement Begin development of Cohort 2 Teacher Guides Continuous Classroom Reading Assessment (CCRA) 1.2.3 Print, package Select and negotiate printing and distribution contract for Cohort 1 and distribute TLAs, TLAs. MLPs, and CCRAs Complete printing and distribution of first generation of Cohort 1 TLAs Intermediate Result 2: Increase Quantity of Reading Instruction in Grades 2-3 in Target Schools 2.1.1 Design, pilot, Conduct SMA enumerator training and implement SMA Carry out SMA baseline data collection in 180 schools instrument 2.1.2 In-service School TOT workshops for SDs and ZIP training coordinators Directors Initial SD training workshop Training/Coaching Complete development of SD training modules Model Begin instituting SD school management meetings every three weeks (based on recommendations from SDs, PDs, and local education
authorities indicating that twice-monthly meetings will create a burden given all other SD responsibilities). 2.1.3 Develop School Complete development of SMRA tool and integrate into SD training Management Rapid Begin development of first generation SMTs Assessment (SMRA) and School Management Tools (SMTs)
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 21
2.1.4 Print, package, Negotiate SMT printing and contract and distribute SMTs
8. Evaluation/ Assessment Update
Evaluations, Assessments, Studies and Audits Include any and all types of evaluations, financial or programmatic, internal or external. Completed: Completed pilot testing of EGRA and SMA Major Findings/Recommendations: N/A instruments Planned: List evaluations, assessments, studies and/or audits planned for next year EGRA and SMA Baseline Data Collection for Impact Evaluation/RCT: • Revise EGRA and SMA instruments according pilot results submitted to USAID • Complete enumerator training • Carry out baseline EGRA and SMA data collection in 180 schools • Create system to collect necessary information for cost-benefit analysis per data requirements to be submitted by IE Contractor
Project Formative Evaluation Tools: • Complete development of SMRA tool • Complete Cohort 1 CCRAs
9. Administrative Update:
Authorization to Operate in Mozambique: During this quarter the process of registration was finalized and WEI was issued an approval to operate in the following provinces: Maputo, Nampula and Zambézia in Mozambique. Due to the many administrative and bureaucratic requirements for this, registration is usually a lengthy process. ApaL’s proactive approach which achieved a rapid completion of the registration process is considered a prominent success. ApaL’s receipt of authorization from MINEC supported a smooth registration process in Nampula and Zambézia provinces, despite initial resistance from local entities.
Office Establishment and Procurement: ApaL has set up all the offices in the provinces and procured the basic assets for the project set-up. Final disposal of JSI Inc. equipment was approved and all equipment was distributed to Nampula and Zambézia (Quelimane) offices as per the plan and need of each province. Equipment was procured for Maputo office, since all which was in it was sent to the provinces.
Staffing: All project expected staff were hired in all provinces and are working 100% time on their areas of responsibilities. The Local Education Institution Capacity Building position has been replaced and additionally the project hired the M&E National Coordinator, both of whom started working in December.
Expatriate Staff Process: The Ministry of labor in Maputo has issued the work permit authorization for the Chief of Party. In the provinces the process it’s still ongoing for the authorization of the USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 22
expatriate work permit and due to the various factors such as unnecessary bureaucratic attitude with the entities related to this process, WEI staff is making an effort to overcome this current situation.
10. Success Stories and photos:
On October 8-12, 2012, ApaL held the first two in a series of three book development workshops. Workshops #1 and #2 are intended to train local teachers and education officials to write from their own experience and knowledge and then revise stories that will help young children learn to read. Although they were held early in project implementation, Workshops #1 and #2 in Nampula province this quarter proved a positive kick-off for ApaL activities in the provinces.
With participants from both Nampula and Zambézia provinces, including local teachers, trainers from the local teacher training colleges, and education officials from local government offices, these workshops introduced ApaL’s approach to reading instruction to a wide range of education stakeholders. In addition, the workshops both involved local stakeholders in a unique and meaningful way and helped instill local support for ApaL’s interventions. Workshop participants were actively engaged in the process and by the end, 27 stories were created.
These stories are now being adapted by reading experts, who are editing them to ensure they are at the right level for students in Grades 2 and 3. They are also working to make sure that each book, in addition to telling a story, contains important literacy and language lessons, which will help students to learn to read. In large classrooms, often without reading materials for children these books will help students master basic reading skills in the early grades, which will support their ability to learn in all their subjects as they advance in their education. In addition, because they have been written by authors who know the local context, these books will contain images, stories, values, and traditions that are familiar to students and thus allow them to make connections between the books and their daily lives, which is an important part in helping children learn to understand what they are reading.
Although the goal of the workshops was to create stories to be developed into reading books, these workshops also provided opportunity and motivation for many of the participants. The new opportunity and new skills helped many participants to truly engage in ApaL’s overall objective— improving children’s ability to read. At the end of the workshop, ApaL completed an anonymous workshop evaluation. Quotes from several respondents in this evaluation are listed below and demonstrate the effect that ApaL’s workshop had in galvanizing interest in and support for ApaL’s interventions. ApaL will hold two further authoring workshops in subsequent quarters to gather stories for Cohort 2 and 3 TLAs. We hope that the continuation of these interesting and exciting opportunities for local teachers, project partners, and education officials will further reinforce the positive impact that ApaL hopes to have on all education stakeholders in the provinces.
“I am very happy to participate in the workshop which gave an opportunity to learn how to write stories for pupils in 2nd and 3rd grade, and also, it allowed me to interact professionally with my colleagues from Nampula and Zambézia province. Personally, I felt, I learned something new - a skill which I can apply in the classroom.”
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 23
“The experience of becoming a writer for a week was phenomenal. It was a short, but a very good experience which gave me a great motivation to write. The workshop was very productive, as I learned many techniques and in particular writing books for children.”
“From the professional point of view, the workshop taught me how to work with pupils with different kinds of text and how to include these approaches in my practice as a teacher. I will be using the tools in my future practice. During the workshop I have realized many challenges that need to be addressed when teaching to read and write.”
Photos Enumerator Training for EGRA and SMA pilot
EGRA and SMA Pilot Testing Activity
Task Analysis with SDs and PDs
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 24
Book Authoring Workshop
11. Financial Information:
Summary Financial Report: (for full details, please reference attached financial report)
Current Period Total Expenditure: $573,370.92
Following Period Expected Expenditure: $876,284.18
USAID/APRENDER A LER Contract No. AID-656-C-12-00001 FY2013 Q1 Progress Report Page | 25