1 Heading Level One

Government of The Gambia Department of State Education Department For International Development, UK

The Gambia: Basic Support for Poverty Reduction (BESPOR)

DRAFT DOCUMENT 02

Report of the Advisers: Monitoring and Evaluation (Baseline Studies)

December 2005 The Government of The Gambia: BESPOR Project

Report Distribution and Revision Sheet

Project Name: BESPOR Project Code: XX BA01

Report No.:

Report Title: Report of the Advisers: Monitoring and Evaluation

Rev Date Originators Checker Approver Nancy Yildiz Yang Min

"This document has been prepared for the titled project or named part thereof and should not be relied on or used for any other project without an independent check being carried out as to its suitability and prior written authority of Cambridge Education Consultants Ltd. (CEC) being obtained. Cambridge Education Consultants Ltd. (CEC) accepts no responsibility or liability for the consequences of this document being used for a purpose other than the purpose for which it was commissioned. Any person using or relying on the document for such other purposes agrees, and will by such use and reliance be taken to confirm his agreement to indemnify Cambridge Education Consultants Ltd. (CEC) for all loss and damage resulting therefrom. Cambridge Education Consultants Ltd. (CEC) accepts no responsibility or liability for this document to any party other than the person by whom it was commissioned."

"To the extent that this report is based on information supplied by other parties, Cambridge Education Consultants Ltd. (CEC) accepts no liability for any loss or damage suffered by the client, whether contractual or tortious, stemming from any conclusions based on data supplied by parties other than Cambridge Education Consultants Ltd. (CEC) and used by Cambridge Education Consultants Ltd. (CEC) in preparing this report."

2 The Government of The Gambia: BESPOR Project

LIST OF CONTENTS Executive Summary

1. List of Agreed Actions

2. Aims of the Consultancy Visit

3. Main Activities

3.1 Strand 3 Baseline Study of Teaching and Learning Conditions 3.1.1 Rapid Appraisal in Region 5 3.1.2 Development of Survey Instruments 3.1.3 Piloting of Survey Instruments in Region 5 3.1.4 Preparation for Baseline Data Collection 3.1.5 Training of Data Collectors and Data Collection – January 2006 3.1.6 Technical Support for Data Analysis and Interpretation 3.1.7 Logistics and Budgets 3.1.8 Capacity Development of TLC Team

3.2 Update on Strand 1 – EFA Assessment

3.3 Update on Strand 2- National Assessment Tests for Grades 3 and 5

3.4 Update on Strand 4 – Longitudinal School-based Case Studies

3.5 Next Steps.

ANNEXES Annex 1: Terms of Reference for M&E Consultant Visit Annex 2: Schedule of Visits Annex 3: Persons Consulted Annex 4: TORs for Gender Consultant Annex 5: TLC Team Members Annex 6: Workshop Schedule and Agenda Annex 7: Schedule and Logistics for Trialling Instruments in Region 5 Annex 8: School ID Codes Annex 9: ID codes for Teachers, Pupils and Communities Annex 10: List of Potential Cluster Monitors to be Data Collectors Annex 11: TORs for Data Analyst Annex 12: Draft Budget for Data Collection Annex 13: TORs for International Consultant

Acronyms

BCS Basic Cycle School BESPOR Basic Education Support to Poverty Reduction CREDIT Curriculum, Research Directorate and Information Technology CTL Conditions of Teaching and Learning CM Cluster Monitor DC Data Collector DH Deputy Head DoSE Department of State Education EFA Education for All EMIS Education Management Information System HT Head Teacher ITC Information Technology, Communication LBS Lower Basic School

M&E Monitoring and Evaluation PTAC Parent Teacher Association Committee SQAD Standards, Quality Assurance Directorate TORs Terms of Reference RED Regional Education Director REO Regional Education Office# UBS Upper Basic School VSO Volunteer Service Overseas WAEC West African Examinations Council WSD Whole School Development

EXECUTIVE SUMMARY

The International M&E Consultant visited BESPOR Project between November 22nd and December 16th, 2005 to facilitate the development of the Strand 3 Baseline Study and to review preparatory activities for Strands 2 and 4.

Substantial progress was made in developing and trialling Strand 3 baseline instruments that will be used to collect data on teaching and learning conditions in Region 5. There was strong commitment and participation of key stakeholders from the Department of State Education at the national and regional levels as well as from the BESPOR National Consultant for Gender and Community Participation. The key outputs of the three week assignments were:

1. the formation of a core team of six members to support the development of the baseline instruments with representation from Region 5 and DoSE headquarters; 2. increased awareness of the importance of baseline research among the TLC Core team members and their increased capacity to conduct rapid appraisals of schools and develop appropriate baseline instruments that reflect an integrated approach to whole school development; 3. the development and trialling of eight survey instruments to be used in focussed interviews, either one-on-one or in small groups. Separate instruments were designed for head and deputy teachers, LBS and UBS teachers, pupils in Grades 5/6 pupils, PTA Executive Committee, and the wider community. School information sheet and observation forms were also developed to support the data collection process and analysis; and 4. the development of selection criteria for data collectors and terms of reference for the data analyst.

Discussions on data collection timelines and procedures and training of data collectors were held with various stakeholders with the following recommendations: - five teams, comprised of five members of which two are female, should be organized to collect the data in 40 schools in Region 5; - intensive 5-day training of data collectors and additional two days for team leaders should be mounted in early February in Region 5 prior to the data collection; - data collection should be carried out over a ten-day period in February with each team being responsible for eight schools and spending one full day in each school and its communities; - the selection of data collectors will rest with DoSE at the national for those persons outside of Region 5 and with the Director of Region 5 for those within the Region; - the logistics for data training and data collection will be the responsibility of Region 5 in consultation with those persons at DoSE responsible for Strand 3 and from BESPOR Management; - a circular should be sent out to all schools in Region 5 informing them of the forthcoming baseline study. The schools will be selected randomly except for BESPOR schools and teams will arrive unannounced; and - the budget would be revisited and fine-tuned to reflect the changing scope of the baseline study and the need for logistic support for five teams, all collecting data at the same time.

The proposed schedule and tasks for moving forward with Strand 3 are detailed in Section 1.0.

Limited time did not permit an extensive review of progress of Strand 2. Meetings with the World Bank team and BESPOR resulted in the identification of a series of steps to ensure that the national assessment testing is on schedule and that WAEC has the capacity and resources to meet its commitments. These include the need to: form a management 5 The Government of The Gambia: BESPOR Project committee to oversee the logistics and process, moving the trialling dates ahead to allow more time to prepare, getting the support of SMT and GTU to test teachers’ knowledge of the subject matter, and mounting a campaign to increase awareness of DoSE’s plans to test Grades 3 and 5 pupils nationwide.

For Strand 4, preparation has started with the involvement of the Director of Tertiary/Higher education in Strand 3 data collection to prepare the way for this longitudinal study. The plan to track a selected number of BESPOR schools in Region 5 remains on track but implementation has been delayed until February-March 2006. Preliminary tasks will include identification of a research team, a workshop on the design and scope of the study and training of team members. This will take place in March with data collection now re: scheduled for April 2006.

Concerns:

There is an on-going need to revise and/or fine-tune the budgets before disbursement of funds. This requires high level management decisions on per diem rates and honoraria for resource persons before this can happen. E.g. decisions are whether the FTI rate of D588 should be followed or adjusted upwards due to higher accommodation and meals costs needs to be reviewed. Similarly, guidelines for payment of honoraria to resource persons and costing out of other budget items needs to be clarified; e.g. costs related to rental of workshop space, fuel, supplies, printing, photocopying, etc. Access to this information will better assist Consultants to prepare budgets to the level of detail required by BESOR Management.

The proposed baseline studies require a considerable amount of technical support from the M&E Consultants and to a lesser extent from the National Gender Consultant. This can quickly use up the number of allotted days for Consultants. BESPOR management will need to make strategic decisions on how best to do utilize the Consultants’ time and in what capacity. This report recommends that wherever possible the administration of the baseline studies, including logistics, be left to DoSE and WAEC. The M&E Consultants should concentrate on providing focussed technical support and training and avoid duplication of efforts wherever possible.

For Strand 3, the next inputs by the International Consultant will focus on finalizing the baseline instruments, the development of training materials, the training of data collectors, and working with the data analyst on the kind of data analysis required. This will require three additional days to pull the instruments together before the end of December for review by other BESPOR Consultants as well as up to 27 days in the field for the activities mentioned above. Facilitating data analysis and reporting will require up to 14 days. The support of the National Consultant on Gender is required to finalize the community survey instruments (3 days), training of data collectors in using participatory approaches with the communities (up to 10 days incl. travel) and for data interpretation and reporting of Strand 3 findings (5 or more days). The number of days for both consultants is relatively high and alternative solutions may need to be found to support the data collection process and writing up of the results. Suggestions are most welcomed.

1.0 SCHEDULE AND LIST OF AGREED TASKS

Strand 3 Baseline Study

Dates Activities

By Dec. 31, 2005 Completing Baseline Instruments – 3 days – N. Yildiz By Jan. 6, 2006 Review of Instruments by BESPOR Consultants (1/2 day) Jan 13 –Feb 10th Nancy Yildiz in The Gambia (24 to 27 days incl. travel days)I By Jan. 16 Circular to RED 5 Schools of Forthcoming Baseline Survey By Jan. 16 Identification of Data Collectors by DoSE and RED 5 By Jan. 16 Revised Budget for Strand 3 By Jan. 20 Logistics Planning for Training of Data Collectors and Data Collection (M. Sowe, Mr. Swareh and Mr. Bah RED 5) Jan. 16-17-18 Review Baseline Instruments by Core Team with N. Yildiz (3 days) Jan. 19-26 Printing of Baseline Instruments (at least 7 days) Jan. 19-26 Preparation of Training Materials for Data Collectors and Other Materials for Data Collection . N. Yildiz with O. Jatta, M. Sowe and I. Touray (PT) Jan. 24 Meeting with M&E Management Team to report on Progress Jan. 23-Feb.3 Design of Database by Data Analyst (2 weeks?) Jan. 28 Travel to RED 5 --N. Yildiz, M. Sowe, I. Touray Jan. 29,30 Training of Team Leaders Jan. 31st Preparation of Team Survey Packages Jan. 31st Data Collectors travel to RED 5 Feb 1-5 Training of Data Collectors Feb. 5th P.M. Teams travel to their base camps to start data collection Feb 6th Full Day Trial Run for Each Team Feb. 7-16 Data Collection -- over 10 days with Feb. 12th as day off Feb. 17 Debriefing of Data Collectors - one day Feb. 18 Data Collectors return to their respective homes Feb. 20 Mar.06 Data Entry and Cleaning -- up to 2 weeks Mar. 6-19 Data Analysis– up to 2 weeks Mar. 20-31 Data Interpretation and Report Writing -- up to 10 days for TLC Team with Consultants.

Strand 3: Survey of Teaching and Learning Conditions

No. Tasks Timeline Persons Responsible 1.0 Awareness Campaign and Support 1.1 Discuss proposed Baselines Studies with As soon as BESPOR Team Leader SMT and CCM and get their endorsement. possible 1.2 Build awareness of BESPOR in RED 5 and Before the Nancy to draft circular, forthcoming survey of teaching and learning end of term E. Petersen to fax to conditions that will take place in February Mr. Swareh for 2006. circulation to all schools 2.0 Budget for Strand 3: Data Collection 2.1 Confirm DSA Rates for team leaders and As soon as E. Petersen with Mr. data collectors as well as prepare written possible Sanneh guidelines re: when and to whom honoraria should be paid. 2.2 Prepare a detailed budget for Strand 3 to Before 11th Draft: N Yildiz cover all activities related to finalization of of January, Finalize:M. Sowe with instruments, data training and data collection. 2006 M. Sonko, E. Petersen

No. Tasks Timeline Persons Responsible

2.3 Prepare a mini-budget for training activities Before 11th Mr. Swareh with input and data collection being conducted in RED 5 of Jan. from M. Sowe 2.4 Budget Approved By 13th of E. Petersen Jan. 2006 3.0 Formation of Data Collection Team 3.1 Identify possible data collectors in By 13th of DoSE: M. Sowe with consultation with DoSE and The Director of Jan. Omar Jatta RED following the criteria agreed on (see Annex xx). 3.2 Screen potential candidates where necessary By 13th of M. Sowe with Omar and select a core team of 20 data collectors. January, Jatta at national level. RED 5 to select 7 of the 20 persons from that 2006 Mr. Swareh for RED 5 that region. 3.3 Confirm data collection team and inform them By 16 of M. Sowe, Mr. Swareh in writing of full schedule (training dates, data January, in consultation with collection timelines, etc) 2006 Mr.Sanneh and Erling Petersen 4.0 Preparation of Baseline Instruments 4.1 Complete TLC Surveys with Crosschecks By Dec. 31, N. Yildiz (3 -4 days) (unable to finish for this consultancy) 2005 4.2 BESPOR Consultants to review and give By 6th of N. Santcross, M. comments Jan. 2006 Young, G. Terry 4.3 TLC Team to review second drafts and Jan 16-18 TLC Core Team with N. triangulate data, finalize instruments Yildiz 4.4 Print sufficient number of Instruments to Jan. 19-26 BESPOR cover 40 schools (quality of printing must be Administration good) 4.5 Meeting with M&E Management Team and January 24t M. Sonko to organize other key stakeholders to Baseline Study progress 5.0 Training of Data Collectors 5.1 Arrange logistics for training workshop for By 25th of Mr. Suwareh with Musa data collectors and base camps for data Jan. 2006 Sowe, Omar Jatta and collection. Lilian Breakell 5.2 Prepare Training Materials for Data Jan. 19-26 N. Yildiz, O. Jatta, M. Collectors (incl. Printing) and Guidelines for Sowe, (5 days) I. Data Collection Touray (2 days) 5.3 Preparation of Supplies to go up-country Jan. 19-26 BESPOR Admin

5.4 Travel to Region 5 Jan. 28 TLC Team N. Yildiz, I. Touray 5.5 Training of Team Leaders (Core TLC Team) Jan 29-30 TCL Team with N. Yildiz and I. Touray

5.6 Preparation of Materials for the Five Survey Jan. 31st TCL Team with N. Teams Yildiz 5.7 Conduct Training of Data Collectors Feb. 1-5 TCL Team with N. (Workshops and Practical Field Trips) (5 days) Yildiz, I. Touray 5.8 Data Collectors Travel to Base Camp near Feb. 5 p.m. Each DC Team Clusters they will monitor 8 The Government of The Gambia: BESPOR Project

No. Tasks Timeline Persons Responsible 5.9 Each team to trial instruments in a school for Feb. 6th Each DC Team a full day 5.9 Data Collection (5 Teams will cover 10 Feb 7-16 Each DC Team clusters –each team to cover 8 schools) 5.10 Bring completed survey forms into RED 5 Feb. 17 M. Sonko and M. Sowe and hold debriefing meeting with DC Teams 5.11 Data Collectors to travel their homes/regions Feb 18 All involved

6.0 Data Entry and Data Analysis 6.1 Confirm Data Analyst and prepare Contract By 20 of M. Sonko Jan. 2006 E. Petersen 6.2 Design database and Master Link files Jan. 23- Data Analyst Feb. 3 6.3 Identify Kind of Analysis Required Feb. 7-8 M. Sonko and N. Yildiz with Data Analyst 6.4 Data Entry and Cleaning Feb. 20-Mr. Data Analyst 6 6.5 Data Analysis Mar. 6-19 Data Analyst

6.6 Data Interpretation and Report writing March 20- Data Analyst, TLC 31 Team with M. Sonko, N. Yildiz and I. Touray (PT)

Strand 2 National Assessment Test Timelines Responsible 2.1 Mount an awareness campaign of the As soon as E. Petersen with forthcoming national assessment test of possible DoSE Planning and Grades 3 and 5 through SMT and CCM WAEC 2.2 Get support of DoSE and Gambia As soon as E. Petersen, M. Sonko Teacher’s Union to include an assessment possible with M Jallow of DoSE of Grade 3 and 5 teachers as part of the Planning national assessment testing 2.3 Establish a Management Committee for As soon as E. Petersen with M. National Assessment Test lead by DoSE. possible Sonko with DoSE Planning 2.4 Discuss with WAEC sampling procedures: As soon as TBA (Consultant from e.g. sampling schools or pupils possible World Bank 2.5 Discuss revision of timelines for trialling of As soon as Malamin Sonko (?) in test items (aim for Feb)and administration possible consultation with of national test (aim for end of May BESPOR TL beginning of June) 2.6 Review data collection forms for pupils and By 20th of Jan. M. Sonko, N. Yildiz teachers that were drafted by WAEC and 2006 give feedback 2.8 Develop an Implementation Plan for By 20 Jan WAEC with support National Assessment with a revised budget 2006 from Malamin Sonko that is more detailed. Also develop a mini- and/or World Bank budget for trial testing 2.9 Approve budgets By 30 of BESPOR TL January World Bank?? 2.10 Trial Testing Late Feb- WAEC in collaboration 9 The Government of The Gambia: BESPOR Project

March 2006 with DoSE 2.11 Revisions and Finalize of Tests By 30 March WAEC 2.12 Printing of Tests By end of April WAEC 2.13 Logistics for Nationwide Assessment By mid April WAEC in collaboration with DoSE 2.14 Conduct National Assessment Tests Late-May and WAEC with DoSE Early June 2.15 Data Entry and Cleaning June-July WAEC 2.16 Data Analysis and Reporting August- WAEC September

Strand 4: Longitudinal School-based Case Timelines Responsible Study 2.1 Following Strand 3 data collection, mount As soon as M. Sowe with Mr. an awareness campaign among BESPOR possible Swareh of RED 5 Target schools of this longitudinal study. 2.2 Revise timelines for Strand 4 As soon as M. Sowe possible 2.3 Workshop to identify scope of study and By Feb. 2006 M. Sowe, M. Sonko, N. criteria to select schools and cohorts to be Yildiz involved in the study 2.4 Identify core team members for Strand 4 By Feb. 2006 M. Sowe in consultation with N. Yildiz, M. Sonko 2.5 Draft a two-week training programme By Feb. 2006 M. Sowe in consultation with N. Yildiz and M. Sonko

2.0 AIMS OF THE CONSULTANCY

The M&E International Consultant visited BESPOR between the 22nd of November and 16th of December, 2005 to support the planning and implementation of baseline studies for Strands 3 and 4 and report on progress of Strand 2. Tasks included:

1. development and trialling of Strand 3 instruments; 2. planning the next steps for data collection and analysis of Strand 3; 3. input into preparatory activities for Strand 4 – the qualitative case studies; and 4. updating of tasks and timelines for Strand 2 –National Assessment Tests.

During the visit, relevant project documents were reviewed and working sessions were held with key education stakeholders who were or would be involved in the baseline studies. The National M&E Consultant, Mr. Malamin Sonko, coordinated the logistics and the National Gender Consultant, Dr. Isatou Touray was given a short-term contract of nineteen days to support the development of baseline instruments and address issues related to gender mainstreaming.

The terms of reference are in Annex 1, a schedule of visits in Annex 2, and persons consulted with their contact information is in Annex 3. TORs for the Gender Consultant are in Annex 4.

1.1 MAIN ACTIVITIES OF THE CONSULTANCY

3.0 Strand 3: Survey of Teaching and Learning Conditions

The major thrust of this consultancy was to design and trial baseline instruments for Strand 3. These instruments will be used to collect data on teaching and learning conditions in Region 5 in January 2006. The development process took place over three week period, November 23rd to December 15th, 2005.

The original plan to have DoSE/SQAD or Gambia College lead this strand had not been resolved at the time of this visit. This resulted in the International M&E Consultant, N.Yildiz, taking the lead with support from the National Gender Consultant, Dr. Isatou Touray, and the National SQAD officer, Mr. Omar Jatta. The core team was joined by three education staff from Region 5 – one Senior Education Officer, one Upper Basic School Principal and one VSO Volunteer. See Annex 5 for a list of names of the TLC Team.

3.0.1 Rapid Appraisal in Region 5

The starting point to develop the baseline instruments was a review of the proposed whole school development (WSD) model. Key areas with possible education indicators were identified along with a host of possible questions/topics to consider. These key areas reflected the six major components of WSD. The indicators were aligned with the areas that the Cluster Monitors are responsible for monitoring. This working document was used as a guideline for the rapid appraisal of six schools.

These schools in the Central River Division (Region 5) were visited between November 30 and December 1st. The purpose was to gather preliminary data that would help the team develop the survey questionnaires. See Table 1 for a list of schools. All six TCL members participated in this activity over a four day period. The Team visited each school for a half- day. One on one Interviews were held with head teachers, deputies, and a cross-section of teachers (qualified, unqualified, and teacher trainees). Small group discussions were held with PTA members, parents and pupils, both boys and girls. Many of the interviews touched

11 The Government of The Gambia: BESPOR Project on the same questions in order to cross-check information and identify possible areas for triangulation of data of the different stakeholders.

Table 1: Schools Visited During Rapid Appraisal

1 Kerewan Samba Sera LBS 2 Jammally LBS 3 Nyanga Bantang BCS 4 Sambang UBS` 5 Kudang UBS 6 Njau LBS

The schools selected for this exercise were not the schools that were targeted for Phase 1 of BESPOR. The BESPOR target schools will be done during the actual data collection in February 2006.

The outcomes of this rapid assessment of schools helped to prioritize the areas where relevant data could be collected and identify factors that the team needed to consider when developing the survey instruments. The key findings directed the team to pay close attention to the following areas; e.g.

Leadership and Management

The extent to which: - The school has created a safe and secure environment - School policies are clearly understood and jointly developed with key stakeholders - Senior Managers are trained in school management - School-based supervision and monitoring are happening - Routine school records (e.g. timetable, attendance, supplies etc.) are up-to-date - PTACs are active with transparent decision-making process and financial accounting

Professional Development of Teachers

The extent to which: - Teachers are active participants in the development of the school and PTACs. - In-service training supports the professional development of teachers - Teacher Trainees are receiving support from their mentors and coping with the distance education assignments - Punctuality and mobility of teachers impacts on pupil contact hours and the quality of teaching and learning; - Teacher’s welfare (basic needs for housing, salaries on time) impact on their teaching

Curriculum Coverage and Teaching and Learning Resources

The extent to which: - The curriculum is being covered within the allotted timeframe and factors that impede complete coverage - Access to and timely delivery of teaching resources, textbooks and supplies

Pupil Welfare

The extent to which:

- Pupils are marginalized because of poverty (no books or lunch given if pupils have not paid) - Corporal punishment, sexual harassment and other forms of punishment exist in schools - Pupils’ have access to a well-balanced diet

Community Participation and Support

The extent to which: - Parents are aware of and support the development of the school; - Community views on SFP, corporal punishment, pupils with SEN are integrated into regular classes

Communication and Working Relationships

The extent to which: - The working relationships and support mechanism among all key stakeholders is supporting or hindering the school’s development (Cluster Monitors and Schools, School Managers and teachers, schools and PTACs/Communities) - Cluster Monitors are monitoring and supporting the school development process - The flow of information to/from the Regional Office to schools, teachers communities is effective

3.1.2 Development of Survey Instruments

Following the rapid appraisal, the TCL Team developed the survey instruments. This took place between December 3rd and 8th at the Sunset Beach Hotel in Banjul. A copy of the timetable and activities is in Annex 6. The outcomes of that workshop was a first draft of eight baseline survey instruments as per Table 2. Each instrument was designed to be conducted as a focussed interview. Prompts for the Data Collector were included in each survey form as well as a checklist of documents to be observed after the interview. These will require further revision. Table 2 Instruments for Baseline Survey On Teaching and Learning Conditions

No. Survey Instruments 1 Head Teacher/ Deputy Head Survey 2 LBS Teacher Survey 3 UBS Teacher Survey 4 BCS Teacher Survey 5 LBS Pupil Survey 6 UBS Pupil Survey 7 PTAC Survey 8 Community Survey Other Baseline Forms Doc 1 School Information Sheet Doc 2 School Observation Form Doc 3 LBS Enrolment/Attendance Data Form Doc 4 UBS Enrolment/Attendance Data Form Doc 5 Teacher Attendance Form

A school information sheet and school observation forms were developed at the same time. The information sheet collects pertinent background information on the school similar to

EMIS data. The observation forms assess facilities (e.g. dangerous classrooms), furnishings (desks/chairs), teaching and learning resources but in less detail than EMIS. A decision was taken not to count books and desk/chairs but to identify which of these items were adequate, inadequate, or severely inadequate. This was based on ratios of number of pupils per desk and number of books shared among pupils.

A decision was also taken to collect attendance data on pupils and teachers. This was done by looking at school and class registers and the teacher attendance book over a specified time period. It is expected that an analysis of teachers and pupils not reporting on time after the holidays will shed some light on the reduction of pupil contact hours. Analysis of pupils’ attendance should provide preliminary data on dropouts which was not being collected by EMIS at this point in time.

3.1.3 Piloting of Survey Instruments in Region 5

The trialling of the instruments took place from December 12th to the 14th and involved two teams. Team 1 visited three schools on the south bank and Team 2 visited three schools on the north bank. Schedule and other logistic information are in Annex 7. Schools visited were: Table 3: Trialling of Instruments

Team 1 (South Bank Team 2 (North Bank Njoren LBS Demfai Njaga LBS Sare Njai BCS Jarumeh Koto BC Bansang UBS Kuntaru UBS

The teams were joined by Musa Sowe, the Director of DoSE/Higher Education. His participation had two objectives. One was to be a floater and observe how the data collection process was being implemented and could be improved. The second was to familiarize himself with the kind of data being collected as a preparatory step for developing Strand 4 – the longitudinal study of ten BESPOR target schools.

Schools were not forewarned of the team’s visit to avoid contamination of data and schools prepping themselves to look good. Many schools did not know their EMIS ID Code so a special ID Codes were assigned all schools in Region 5 for Strand 3. A master list can be found in Annex 8. The codes indicate the region +cluster number + school number within cluster. Schools that are bolded on the list have been identified as the BESPOR Target Schools in Phase 1. For data collection purposes, the ID codes for head teachers, teachers, pupils, will be preceded by the school code for ease of identification. Examples of these codes are in Annex 9.

The trialling of the instruments was very intensive and went well. Each team spent a full-day in each school. In the late afternoon, team debriefings were held back in Janjanbureh to review and recommend any changes to the instruments and to data collecting procedures. Where time permitted, changes were made to the instruments and a second draft was produced. Outcomes of this process included the following recommendations:

1. the need for an awareness campaign as most head teachers interviewed knew very little about the forthcoming BESPOR Project; 2. Cluster Monitors (CMs) from Region 5 should not be data collectors as it would be difficult for them to be both an interviewer as well as an interviewee (conflicting roles). These CMs could possibly play a logistics role to support the data collection teams in their respective clusters; 3. Each data collection team should comprise of at least 4 members and preferably five with two of the members being female; and 14 The Government of The Gambia: BESPOR Project

4. Selection criteria for Data Collectors (DCs) should take into account that persons appointed should: - include a wide range of representative stakeholders from national, regional and school levels; e.g. CMs from other regions, DoSE/SQAD staff, Gambia College instructors, GTU, as well principals and teachers from within RED 5. The proposed breakdown would be 7 DCs from RED 5 excluding the Team Leaders (M. Bah, A. Mendy and Fiona). The other 13 DCs (excluding O. Jatta and M. Sowe) would come from outside the region; - not include teaching staff from target BESPOR Schools in RED 5; - be able to speak the local languages as most parent and pupils’ comprehension of English is too low to conduct the survey in English; - be familiar with the lower and upper basic education system in The Gambia; - have patience and tolerance when collecting data; and - have some experience conducting interviews for research purposes.

Note: This report recommends that the selection of Cluster Monitors from other region should be based on the assessment and outcomes of the Cluster Monitor Training Workshop held the week of November 28th through to December 2nd, in Region 4. A potential list of highly qualified CMs has been made available by the BESPOR Consultant who conducted the workshop and can be found in Annex 10.

3.1.4 Preparation for Baseline Data Collection

The next steps are to finalize the instruments and this is a four step process.

1. The International M&E Consultant needs to review, cross-check and format the second drafts of the instruments and this will take an additional three to four days. There was insufficient time to do this during this consultancy. 2. BESPOR Consultants will be invited to review the instruments and give their comments; 3. The TLC team will meet to review the drafts and comments and to check that the questions asked allow the researchers to triangulate data across key areas; and 4. A half-day working session will be held with M&E Management Committee and other selected stakeholders to review the plans for this baseline study.

Note: The baseline instruments will be kept under tight security to avoid leakage and problems.

3.15 Training of Data Collectors and Data Collection

The original intent to conduct the baseline data collection in mid-January is not possible. Discussions with the BESPOR Team Leader indicated that key education officers will be heavily involved in regional planning exercises the week of January 16-20. This will delay the data collection until the first two weeks of February 2006.

The revised plan is to develop the training materials and data collection procedures in January, train the data collectors in the first week of February, and collect the data over a ten-day period immediately following the training. See Section 1.0 for a list of agreed tasks and proposed schedule.

Data collectors will require intensive training to ensure quality control of the data being collected. This training will require a minimum of five days for the data collectors and an additional two days for the team leaders. Training will cover interviewing techniques, time management, familiarization with the survey forms, use of local language in interviews, as well as data collection procedures. Facilitators will be N. Yildiz, I. Touray and M. Sowe.

There will be five survey teams to collect the data. Each survey team will comprise of five persons and be led by one of the core TLC members. Selection of team members would follow the criteria mentioned above and would be carried out by Mr. Musa Sowe and Mr. Omar Jatta of DoSE for those DCs coming from outside RED 5. Mr. Swareh, the Director of Region 5, will do it for those persons coming from within the region.

Schools involved in the data collection in January will not be forewarned of the day of the baseline team’s visit. However, this report recommends that the RED 5 inform all schools that a baseline survey for BESPOR will take place sometime in the month of February with no specific date given and that schools will be randomly selected.

During the data collection, Mr. Musa Sowe will supervise the five teams, visiting each team at least twice and troubleshooting where required. Debriefing will be held in RED 5 and will be lead by M. Sonko, the National M&E Consultant. He will also be responsible for ensuring that the survey data is delivery safely to the Data Analyst.

3.1.6 Technical Support for Data Analysis and Interpretation

Critical to the analysis is the design of a database for data entry and analysis. Two potential candidates were interviewed by the M&E Consultants. (See Annex 11 for TORs). The stronger of the two, is Lamin Janneh, and hence the M&E Consultants recommend that BESPOR/DoSE give serious consideration to hiring him. However, the number of days required to fulfil the Terms of Reference is difficult to determine until the survey instruments are finalized and the data analyst has an opportunity to review the scope of work required.

The plan is to design the database in January 2006, enter and analyze the data in late February and early March and do data interpretation and report writing in the latter part of March. The TLC team would be involved in the latter activity with the support of the International and National M&E Consultants and inputs from the National Gender Consultant.

It should be noted that the budget for Strand 3 did not include line items for hiring a Data Analyst or the cost of purchasing Version 12 of SPSS. As such, it is recommended that the budget be revisited and necessarily adjustments made to include these aspects.

3.1.7 Logistics and Budgets

To ensure sufficient funds for the above activities, a preliminary budget has been drafted. This requires further input and review by BESPOR Team Leader and DoSE. (See Annex 12). The issue of per diem rates and whether or not honoraria will be paid to resource persons and data collectors needs to be decided before the budget can be finalized. Note: for the TLC Team, the per diem rates were raised from D588 (FTI rate) to D750 to cover higher costs of accommodation and meals in Region 5 and in Banjul. In addition, each TLC Team member was paid an honorarium of D500 per day in recognition of his/her contributions as a resource person. These rates were approved by the BESPOR Team Leader on a temporary basis.

3.1.8 Capacity Building of TLC Team

It was evident that the whole exercise from the rapid appraisal of schools to develop and trialling of instruments provided an excellent opportunity to build capacity within DoSE at the national and regional level. Every team member felt that s/he had grown professionally in a new directions. Mr. Bah of RED 5 said that prior to this experience he had rarely ventured into the communities. This exercise taught him that communities have a wealth of ideas and opinions about the management of schools and they need to be heard. For Mr. O. Jata of 16 The Government of The Gambia: BESPOR Project

DoSE/SQAD , the exercise brought home the importance of improving the quality of teaching and learning through research, which in turn, can inform policy decisions. Mr. A. Mendy, a principal of an upper basic school realized important structures and systems that were missing in his school and has taken steps to correct this in his school. Fiona Stevenson, the VSO Volunteer, experienced a sharp learning curve as well. The exercise allowed her opportunities to quicken her understanding of the dire situation in schools outside her cluster area and identify key factors that continue to impact on the quality of teaching and learning.

3.2. Update on Strand 1: EFA Assessment of EMIS Data

The plan to do a national EFA Assessment Report for 2001 to 2005 and a sub-sector report for Region 5 remain the same. What has changed is that the World Bank will fund this Strand and provide technical support to the Directorate of Planning to address system problems and conduct training in data analysis and interpretation. This is scheduled to begin in January 2006. At the time of this report, EMIS data for the 2005-2006 academic year was being collected. Further details were not available.

3.3 Update on Strand 2: National Assessment Testing of Grade 3 and 5 Students

Preparation for the National Assessment Testing of Grade 3 and 5 pupils is well underway. A preliminary field visit to Region 5 was carried out by WAEC in June 2005 followed by one- week training for test item writers in September 2005. Nick Taylor, a consultant in test development with the Joint Education Trust (JET) in South Africa, was contracted to support the development of the test items. The Consultant’s Report and WAEC’s three reports on preparatory activities (e.g. field, visit, alignment of LATs with the national curriculum) are now on file at the BESPOR Office.

The trialling of the test items across the four core subjects, which was scheduled for late September 2005, has been delayed until the first part of 2006. Administration of the national assessment tests to 30% of the Grade 3 and 5 in Gambia is still on track and will take place at the end of this school academic year (e.g. June or July). There are, however, some concerns related to budget, the capacity of WAEC to do the job well, and the whole coordination process; e.g. who will oversee this study and troubleshoot.

Discussions with Mr. Thomas Kellaghan of the World Bank, resulted in the following recommendations. These need to be taken up with DoSE/Planning and WAEC. They are:

1. WAEC should move the trialling of the test items ahead to February 2006 to allow more time to finalize the tests and print sufficient copies for the national assessment; 2. Data Collection should be moved to the end of May or early June to avoid the rains impeding access to schools; 3. The sampling should be done by random sampling of pupils, not schools as was done in the past; 4. The capacity of WAEC to do the data entry and analysis should be assessed and areas identified for support and/or strengthening to ensure a smooth national assessment; 5. The budget for this strand should be revisited and details flushed out as the costing appears to be inadequate and no provision was made for the testing of the teachers. A detailed implementation plan needs to accompany the budget for approval. 6. A DoSE Committee should be struck to coordinate the scheduling, logistics, implementation with WAEC; and 7. Through SMT and CCM, DoSE/WAEC needs to mount an advocacy campaign to let regions know of plans to carry out a national assessment test for Grades 3 and 5.

Drafts of the one-page survey on socio-economic conditions of Grade 3 and 5 pupils and their teachers’ qualifications, etc. have been prepared by WAEC. They still need require 17 The Government of The Gambia: BESPOR Project review and input by DoSE/Planning and BESPOR M&E Consultants. The triangulation of data is expected to yield valuable data on variables that may impact on student performance and compliment the data collected in Strand 3. The survey data will be collected on the same day as the national assessment tests.

The plan to give the Grade 3 and 5 teachers similar subject matter tests may be problematic. The first concern is that the matter requires the support of the Gambia Teachers’ Union. These talks are scheduled to take place over the next few months between DoSE/BESPOR and GTU. The second concern is that if subject matter tests are given to teachers they should reflect a higher level of knowledge than Grade 5. This would require WAEC to add additional test items in the four core subjects and include the costing in its revised budget. The third concern is an advocacy campaign mounted by DoSE to let teachers know why they are being tested and how the results will be used. If the GTU does not agree to a nationwide test for Grade 3 and 5 teachers, then DoSE and BESPOR should make a case to test only teachers in RED 5 as part of the baseline data for the WSD model.

3.4 Update on Strand 4: Longitudinal School-based Case Studies

This strand is still in its infancy stages. The design and scope of this longitudinal study will evolve after Strand 3 data has been collected and analyzed. This will allow Strand 4 to validate Strand 3 data and probe reasons for the state of the conditions found in the schools and communities.

During this visit, preparations were started. Timelines were reviewed with the recommendation to identify the School Case Study Team (SCST) members by February, conduct two-week training course/field visits in March, and collect the data in April 2006. Whether or not these timelines can be adhered to requires input from other stakeholders; (e.g. DoSE and BESPOR management). Mr. Musa Sowe, the Director of Tertiary/Higher Education and Research (THERD) will lead the team. His current input into trialling Strand 3 survey instruments and data collection is providing a solid foundation to move forward with Strand 4.

3.5 Next Steps

The M&E Consultants identified several tasks which will require follow up by DoSE and BESPOR. These are listed in Section 1 – List of Agreed Tasks along with the proposed schedule for Strand 3. The International M&E Consultant plans to return around the 13 of January to facilitate the training of data collectors and data collection. TORs are in Annex 13 and require approval by BESPOR management.

Draft 1: Prepared by N. Yildiz