MSc Project Report 2012-2013
Formative research to scale-up community mental health monitoring and evaluation in Benue State, Nigeria Supervisor: Mary De Silva Co-Supervisor: Alex Cohen
Candidate number: 107467 Word count: 9,867
Submitted in part fulfilment of the requirements for the degree of MSc in Global Mental Health
September 2013
Candidate no. 107467
Acknowledgments
This project would not have been possible without the support of mentors, friends, and colleagues from: London School of Hygiene and Tropical Medicine; CBM International; the Comprehensive Community Mental Health Programme of Benue State; the Methodist Diocese of Otukpo; Makurdi Federal Medical Centre; Benue State Ministry of Health; the Edawu and Agboke community rehabilitation centres; and the local governments and clinic staff of Obi, Agatu, Guma, and Gwer- West.
I would like to give special thanks to my supervisor Mary De Silva, co-supervisor Alex Cohen, and external advisor Julian Eaton, first for the opportunity to get involved in their work in Nigeria, and second for their ongoing support and guidance throughout the duration of the project. My thanks also go out to Lucy Lee and Sujit Rathod for sharing their work on the pilot, and to my tutor Tatiana Salisbury for her support and guidance during the initial stages of project development.
I am very grateful for the warmth, generosity, and involvement of my hosts in Nigeria, especially on the part of Emeka Nwefoh, Sunday Idoke, Samuel Ogbole, and—most of all—Philip Ode and Vivian Odu, who put in at least as much effort to coordinating this project as I did to executing it.
Last but not least, I would like to thank my family—as always—for their steadfast support, and express my gratitude to the staff and students of the 2012-2013 Global Mental Health MSc, who took a chance on a new course and built a community in the process.
i
Candidate no. 107467
Acronyms and abbreviations
CBR Community-based rehabilitation CBM CBM International CCMHP Comprehensive community mental health programme of Benue State CGI-S Clinical Global Impression Severity CGI-SCH Clinical Global Impression for Schizophrenia CPN Community psychiatric nurse GAF Global Assessment of Functioning HIS Health information system MHIS Mental health information system LGA Local government area LMIC Low- and middle-income countries M&E Monitoring and evaluation mhGAP Mental health Gap-Action Programme MOU Memorandum of understanding NGO Non-governmental organisation NKST Nongu u Kristu u i Ser u sha Tar Church PHC Primary health clinic PHQ-9 Patient Health Questionnaire for depression WHO World Health Organisation WPA World Psychiatric Association
ii
Candidate no. 107467
Table of Contents
Acknowledgments ...... i Acronyms and abbreviations ...... ii Table of figures ...... v Executive summary ...... vi 1. Introduction ...... 1 1.1. Health information systems and monitoring and evaluation for community mental health in sub-Saharan Africa ...... 1 1.2. Piloting a monitoring and evaluation system in Nigeria ...... 2 1.3. Adapting the piloted system for Benue State ...... 3 1.4. Designing a mental health information system for Benue State ...... 4 1.5. Formative research to scale-up systems in Benue State ...... 5 1.6. Aims...... 6 1.7. Objectives...... 6 2. Methodology ...... 7 2.1. Design ...... 7 2.2. Data collection ...... 7 2.2.1. Clinical fieldwork ...... 7 2.2.2. Participant observation ...... 8 2.3. Data analysis ...... 8 2.4. Limitations...... 8 2.4.1. Triangulation ...... 8 2.4.2. Positionality ...... 9 2.5. Strengths ...... 9 2.6. Ethics ...... 9 3. Results ...... 10 3.1. The local context: Mental health services in Benue State ...... 10 3.1.1. Private community-based rehabilitation facilities ...... 10 3.1.2. Public clinics in primary health centres ...... 12 3.2. Feasibility, acceptability, and appropriateness of the M&E system ...... 13 3.2.1. Challenges to data collection ...... 14 3.2.2. Challenges to data processing ...... 20 3.2.3. Challenges to data analysis, dissemination, and use ...... 22 4. Discussion ...... 23
iii
Candidate no. 107467
4.1. Implications for M&E system ...... 23 4.2. Implications for MHIS...... 24 5. Recommendations ...... 25 5.1. Changes to processes ...... 26 5.2. Changes to tools ...... 26 5.2.1. Changes to M&E tools ...... 26 5.2.2. Changes to MHIS tools ...... 26 5.3. Next steps ...... 28 6. Conclusion ...... 29 7. References ...... 30 Appendix a. Contextual challenges encountered during pilot...... I Appendix b. Piloted M&E tools tested during formative research ...... II Appendix c. Revised MHIS Tools ...... XVI Appendix d. Permissions ...... XX
iv
Candidate no. 107467
Table of figures
Tables Table 1. Information collected and entered from pilot M&E tools ...... 2 Table 2. Methods used for clinical fieldwork ...... 7 Table 3. Summary of key information on clinics ...... 13 Table 4. Key challenges and recommendations for MHIS and M&E based on findings ...... 25
Figures Figure 1. Process of adapting MHIS and M&E system...... 5 Figure 2. Photograph: Donor report...... 11 Figure 3. Photograph: PHC clinic...... 12 Figure 4. Photograph: CBR clinic 1...... 14 Figure 5. Photograph: CBR outreach clinic...... 15 Figure 6. Photograph: Clinical records...... 16 Figure 7. Photograph: CBR clinic 2...... 17 Figure 8. Recommended system integrating MHIS and M&E...... 27
v
Candidate no. 107467
Executive summary
Introduction
The need to collect high-quality data without overburdening clinicians led to the development and piloting of a new community mental health monitoring and evaluation (M&E) system at two sites in southeast and central Nigeria. This report describes subsequent formative research informing the scale-up of this system for use in Benue State, Nigeria.
Aims
(1) To inform the adaptation of a recently piloted community mental health M&E system for scale- up in Benue State; (2) To inform the integration of the M&E system with a proposed state-wide mental health information system (MHIS).
Objectives
(1) To document the local context in which the piloted system is being proposed for scale-up; (2) To assess the feasibility, acceptability, and appropriateness of the piloted system for the local context; (3) To adapt the piloted M&E tools and procedures, based on findings; (4) To adapt the state mental health information system (MHIS) tools and procedures, based on findings.
Methods
A case study methodology was employed, comprising key informant interviews, document reviews, and participant-observation. A thematic analysis was conducted using Dedoose qualitative research software.
Results
Constraints of service delivery in low-resource, underserved communities negatively impact the feasibility and acceptability of data collection using the pilot system, as well as quality of data collected. Clinicians complain that tools are “bulky” and perhaps inappropriate for services with high patient loads. Routine processing of patient data is not feasible given the existing human resources. Bottlenecks at the data processing stage preclude analysis, dissemination, and use.
Conclusions
Data collection tools are simplified and adapted to serve primarily as clinical records, which then feed into a revised MHIS register and reporting tool. The MHIS is used for monitoring, while clinical records may be audited on a non-routine basis for more thorough formal evaluation. Investment in mobile technologies is recommended to improve the durability of tools for use in communities, and to avoid common issues in data storage, retrieval, and processing.
vi
Candidate no. 107467
1. Introduction
1.1. Health information systems and monitoring and evaluation for community mental health in sub-Saharan Africa
Governments and multilateral organisations are increasingly joined by private sector donors and organisations in the pursuit of better health outcomes worldwide (Szlezák et al 2010). Increased external involvement has amplified existing demands on health systems and programs in low- and middle-income countries (LMIC) to provide accurate and timely statistics for the purposes of monitoring and evaluation (M&E) (Kusek and Rist 2004, WHO 2009, Jenkins et al 2011). Robust M&E improves the transparency and accountability of health programs, aids data-driven decision-making, and facilitates further research (Kusek and Rist 2004). In the field of global mental health, however, M&E remains in its infancy.
The World Health Organisation (WHO) declared “monitor community mental health” one of its top ten recommendations for the emerging field of global mental health in a seminal World Health Report, calling for integration of mental health indicators into health information systems (HIS) in all LMIC (2001). WHO argued that more information could fuel better arguments for investment in mental health services, while improving donor confidence. Subsequent mental health M&E and HIS guides were released to help build the capacity of program implementers and policy- makers to design robust systems (WHO 2005, 2006). WHO thus provided technical guidance and an imperative to improve M&E of community mental health as part of a broader global mental health agenda. Yet mental health indicators have not been integrated into HIS in most LMIC, and M&E remains underdeveloped, especially in sub-Saharan Africa (Jenkins 2003; LaFond and Fields 2003; WHO 2005, 2006; Ndetei and Jenkins 2009; Hanlon, Wondimagegn and Alem 2010; Eaton et al 2011; Jenkins et al 2011).
The frailty of these systems is indicated by the lack of evaluation data available. One review located 24 evaluations of community mental health programmes in sub-Saharan Africa, but 17 of these evaluations were from South Africa alone, and none successfully evaluated clinical outcomes (Hanlon et al 2010). The relative dearth of evidence from low-resource settings is consistent with findings from a recent survey of key informants involved in new mental health programmes in LMIC; less than 40% of respondents reported completed evaluations (Eaton et al 2011). Evaluation of community mental health programmes in low-resource settings is hindered by inadequate systems for generating routine data, especially regarding patient outcomes, as was reported by researchers from the London School of Hygiene and Tropical Medicine (LSHTM) Case Study Project, described below (Cohen et al 2011, 2012).
1
Candidate no. 107467
1.2. Piloting a monitoring and evaluation system in Nigeria
In 2008, researchers launched the LSHTM Case Studies Project in order to develop a systematic methodology for the documentation and evaluation of community mental health programmes in LMIC (Cohen et al 2011, 2012). The resulting methodology was tested at programmes supported by the disability charity CMB international in three low-income settings, including Abuja, Nigeria; Kerala, India; and Tabaco City in the Philippines. During data collection, researchers discovered that none of the three test sites recorded clinical outcomes; hence, the effectiveness of their interventions could not be assessed. To address this issue, researchers expanded the scope of the LSHTM Case Studies Project to include the development of a prospective clinical M&E system that could meet the information needs of service providers, programs, donors, researchers, and other stakeholders, while remaining practicable in a low-resource setting.
The resulting M&E system is comprised of three paper-based data collection tools, including: (1) an enrolment form to be completed for all new patients; (2) a routine follow-up form to be completed for existing patients at most visits; and (3) a more comprehensive six-month review form to be completed biannually for existing patients (see Appendix b.). The tools are used by CPNs during routine service delivery, and act both as patient records to enable clinical decision-making and as sources of process and outcome data for further analysis. A trained programme staff member enters the data from enrolment and six-month review forms into an Access database, which is then transferred electronically to LSHTM for analysis. Individual patient data are linked through a unique patient identification number, allowing key indicators to be analysed over six-month intervals at both the individual and program level. (Table 1. Below compares the general types of data collected and entered for further analysis.)
Table 1. Information collected and entered from pilot M&E tools
Pilot data collection tools Data type Enrolment Six-month review Routine follow-up Administrative Collection Entry Demographic Collection Entry Problem, diagnosis Collection Entry Mental state, Collection functioning, severity Entry Quality of life, social Collection circumstances Entry Rehabilitation plan, Collection side effects Entry
2
Candidate no. 107467
The M&E system was piloted at two CBM sites in central and south-eastern Nigeria, including the community-based rehabilitation (CBR) program in Abuja that had participated in the earlier case study (Cohen et al. 2011). A primary health clinic (PHC) offering mental health services as part of the Amaudo Community Mental Health Programme was also selected (Colwill 1993; Eaton and Agomoh 2008). Each site relies on a CPN to diagnose and treat mental and neurological disorders at the community-level. The system was designed in consultation with the CPNs, in an effort to ensure data collection would not compromise quality of care by either straining patient- provider interactions or adding to the burden of already overextended staff.
Despite these precautions, the quality of the data generated by pilot sites was poor, with large amounts of data missing. Data entry was often severely delayed, despite training and incentives provided to program staff to input information into the database. Neither site chose to continue using the M&E system at the conclusion of the pilot. Although the pilot’s results are still undergoing analysis, anecdotal evidence suggests that issues in the actual design of the tools and database were compounded by much less tractable local contextual factors that ultimately derailed M&E efforts at both sites. (Appendix a. presents an overview of the challenges encountered at each stage in the M&E process, as described during informal interviews with program staff and investigators in the lead-up to this formative research.)
Due to time constraints during mobile outreach, for example, CPNs often completed forms after the clinical encounter, resulting in missing and poor-quality data due to issues of recall. Further, diverse catchment areas present language barriers which, in the absence of appropriate translation, led to missing data. Analysts examined patterns in missing and illogical data to inform further adaptation of the tools’ design; however, they complained of inadequate information regarding the contextual factors that impact data quality. Improving a question’s phrasing does little good for a CPN who cannot translate it into Hausa, or who has no time to record a response. Any adaptation of the M&E system should be informed by further consideration of the local context, beyond initial consultation with CPNs.
1.3. Adapting the piloted system for Benue State
In 2011, a new Comprehensive Community Mental Health Programme (CCMHP) was launched in Benue State as part of a public-private partnership between CBM International, the Benue State Methodist Diocese, and local, state, and federal government. The goal of CCMHP is to scale-up coverage throughout the state by establishing a new mental health clinic in each of the 23 local government areas (LGAs). Services are provided by trained CPNs following WHO mental health Gap Action Programme (mhGAP) guidelines for treatment of priority mental and neurological
3
Candidate no. 107467 disorders (WHO 2010). Clinics operate out of specialized PHCs known as Comprehensive Community Mental Health Centres, which offer an enhanced package of primary care. The first clinic opened in July 2013, while fieldwork for this project was underway. Two more are expected to open before September 2013.
CCMHP also helps to support two private CBR projects founded by the Methodist Diocese in 1996 and 2004, respectively. The CBR projects are based on the Amaudo model, providing biomedical and psychosocial care for a limited number of inpatients with severe mental disorders (Colwill 1993; Eaton and Agomoh 2008). Both projects operate outpatient clinics as well, which offer mostly biomedical treatment to a much larger number of clients. Each CBR project is staffed by a CPN and several non-specialist aides, who operate an outpatient clinic on-site, as well as multiple outreach clinics in churches or other multi-purpose structures in distant LGAs.
Considering the different designs of older CBR clinics versus newer PHC clinics, as well as the many stakeholders to whom CCMHP is accountable—including multiple levels of government, as well as local and international donors—the programme requires a durable M&E system capable of meeting a variety of reporting needs. As discovered during the pilot, this system will fail if it is not acceptable, feasible, and appropriate, given the local context. Further, the system must be integrated with the broader HIS, in order to ensure the long-term sustainability of mental health M&E in Benue State, as will be described further in the following section.
CBM proposed the pilot system as a template to be adapted to the unique requirements of CCMHP for clinical M&E and then integrated into a recently proposed state-wide mental health information system (MHIS), described further below. (Figure 1 illustrates the planned process of adaptation of the HIS and M&E system, respectively.)
1.4. Designing a mental health information system for Benue State
It is best practice to integrate mental health M&E systems into district, state, and national HIS (Jenkins 2003; Jenkins et al. 2011; Ndetei and Jenkins 2009; WHO 2006). Yet Nigeria currently has no mental health information system (MHIS) in place to supply data to the HIS (WHO and MOH 2006). This creates a disincentive for mental health programmes to invest in M&E systems. If a MHIS is eventually implemented, it will render any incompatible systems obsolete.
In an effort to pre-empt future compatibility issues, a partnership between the Ministry of Health, the Benue State Comprehensive Community Mental Health Programme (CCMHP), and CBM International was forged to develop a state MHIS using indicators from the piloted M&E system. The current draft MHIS comprises a daily register and monthly reporting tool, following HIS conventions
4
Candidate no. 107467 for other PHC-level health programmes in Nigeria. The final MHIS will be scaled-up to community mental health clinics state-wide, before being considered for adoption on a national level.
Because of its centrality to the MHIS, shortcomings in the pilot system could have much broader implications for mental health reporting in Nigeria. Further, previous research has shown that the success of implementing MHIS in LMIC is often dependent on local contextual factors which may vary widely between communities (Ndetei and Jenkins 2009). Careful consideration of the local context is therefore essential: first, to inform further adaptation of the pilot system, which may in turn benefit the design of the MHIS; and second, to better adapt the MHIS itself to the context in which it will being implemented.
1.5. Formative research to scale-up systems in Benue State
The M&E system was revised based on Figure 1. Process of adapting MHIS and M&E system findings from the analysis of pilot data, as previously described. However, tools have not yet been fully adapted to reflect the services provided by CPNs in Benue State. CPNs received a half-day training in mid-June 2013 on the use of the tools in their current state, and were encouraged to begin using them in clinical practice as soon as possible. This research was proposed to examine CPNs’ early trial use of the tools and examine the challenges encountered, in order to inform further adaptation of the M&E system. As no formative research or piloting was carried out prior to the initial design of the MHIS, it was expected that findings would also help to inform subsequent MHIS development. Because very little literature exists on mental health M&E in LMIC, this formative research was carried out in the context of an academic study, with the intention of disseminating key results and recommendations to stakeholders in Nigeria as well as to a broader scientific community.
5
Candidate no. 107467
1.6. Aims
The aims of this project are two-fold:
(1) To inform the adaptation of a recently piloted community mental health M&E system for scale- up in Benue State; (2) To inform the integration of the M&E system with a proposed state-level mental health information system (MHIS).
1.7. Objectives
In order to meet the above aims, the objectives are as follows:
(1) To document the local context in which the piloted system is being proposed for scale-up; (2) To assess the feasibility, acceptability, and appropriateness of the piloted system in the local context; (3) To adapt the piloted M&E tools and procedures, based on findings; (4) To adapt the state mental health information system (MHIS) tools and procedures, also based on findings.
6
Candidate no. 107467
2. Methodology
2.1. Design
Because very little information is available on either the context in which the M&E system is being implemented or the stakeholders involved, a case study methodology was adopted. The methodology that resulted from the LSHTM Case Studies Project offers a structured approach to documenting key aspects of a community mental health programme, and was previously tested in Abuja (Cohen et al. 2011, 2012). The methodology for this formative research project was limited to key informant interviews, participant observation, and document reviews, which were conducted with a focus on M&E and potential challenges to the implementation of the new system. Data were collected primarily in the form of field-notes, which were subsequently analysed and used to populate cells of the 15-domain tool provided by the Case Studies Project. M&E tools and record- keeping procedures were also documented photographically at some sites.
2.2. Data collection
Fieldwork took place in Benue State, Nigeria, over a six-week period, from June through July, 2013. Fieldwork included participant observation and interviews at clinics, as well as more general participant observation during CCMHP activities, as described below.
2.2.1. Clinical fieldwork Clinical fieldwork was supervised by a CCMHP staff member, who also provided translation assistance where possible. Because most new mental health clinics had not yet opened in PHCs at the time of fieldwork, much of the data collected comes from CBR clinics. (Table 2 below compares the methods of data collection carried out at each clinic.) Observations were carried out during day- long visits to either the main clinic or an outreach clinic. Interviews with CPNs included a “walk through” of the enrolment tool, in order to identify areas of difficulty or confusion. A follow-up session was also held with each of the three CPNs observed, in order to make further clarifications.
Table 2. Methods used for clinical fieldwork CBR clinics PHC clinics Method Subject Agboke Edawu Agasha Ogabaji Naka Adom-East Interviews CPN Observation CPN Main clinic Users Aides Observation CPN Outreach Users Aides
7
Candidate no. 107467
2.2.2. Participant observation The investigator was heavily engaged in participant-observation with a variety of stakeholders during meetings, routine supervision, advocacy visits, and other programmatic activities carried out by CCMHP. Key stakeholders included: CCMHP staff; the CBM Mental Health Advisor for Nigeria; Welfare Officers from the two CBR facilities; the Bishop of the Benue State Methodist Diocese of Otukpo; the Benue State M&E Officer; the Benue State Director of Public Health; and other state and local government officials.
2.3. Data analysis
A thematic analysis was conducted using Dedoose, an online qualitative research software developed by SocioCultural Research Consultants, LLC. Key themes relating to the research objectives were incorporated into a structured thematic framework, comprising: (1) domains of the Case Study Methodology (Cohen et al. 2012); (2) acceptability, feasibility, and appropriateness; and (3) the five stages of M&E, as described by WHO (2005). Following familiarisation with the texts and in vivo coding, the framework was adapted to include additional themes derived inductively from the dataset. The dataset was then coded using the resulting framework. Results are presented as narratives describing key themes in context, which are further illustrated via excerpts from field- notes, quotes from participants, and photographic documentation, as presented in the results section of this report.
2.4. Limitations
Case studies and qualitative methods more broadly are inherently subjective (Bowling 2009), and the dataset used for this study may have been further influenced by issues of triangulation and the positionality of the investigator, as described further below.
2.4.1. Triangulation Due to constraints of time and access, interviews could only be carried out with the six CPNs who had completed their nursing training at the time of fieldwork. The remainder were engaged in coursework in another state. As only one new clinic was operational at the time of data collection, observations of this setting could not be triangulated among multiple sites.
One of the biggest gaps in this study is the relative lack of participation by service users and caregivers. Benue State is very diverse, and CCMHP staff were unable to provide enough translation support at clinics to facilitate formal interviews. Clinic staff helped to relay some information second-hand from patients, but it was deemed inappropriate for them to translate interviews.
8
Candidate no. 107467
This project relied largely on unstructured methods of interviewing and observation. Although tools were developed for more structured data collection, including a survey of all CPNs trained on M&E, quantitative methods were not deemed appropriate given the small number of CPNs available to participate in this study. Triangulation of methods was therefore limited.
2.4.2. Positionality This study was commissioned by CCMHP, who ensured that the project was cost-neutral to the investigator. The investigator also spent a significant amount of time engaged in participant observation with CCMHP staff and with CPNs. Thus the positionality of the investigator is inclined toward these stakeholders.
Formative research often requires a quick turnaround, and audio-recording and transcription of discussions with key stakeholders were not feasible. The recording of data via field- notes creates an additional opportunity for the investigator’s positionality to influence the dataset.
The presence of CCMHP staff during fieldwork, and of the clinic staff who relayed information from clients, may also have affected how procedures were conducted and what information was disclosed.
2.5. Strengths
The LSHTM Case Study Methodology has already been tested successfully at another community mental health programme in Abuja, Nigeria (Cohen et al. 2011). This research was carried out under the guidance of several of the originators of the methodology and the pilot M&E system, including two LSHTM co-supervisors and an external supervisor from CBM.
A strong working relationship was established with CCMHP, who have a vested interest in the project’s outcomes. Participants were often very candid about the challenges encountered and eager to propose solutions. A participatory approach was taken to the revision of M&E and MHIS tools and processes, with input from CCMHP staff, CPNs, and the Benue State M&E Officer. Outputs from this research have immediate applicability to community mental health programmes in Benue State, and may also be applicable to other CBM-supported sites in Nigeria and in other LMIC.
2.6. Ethics
This project received ethical clearance from the LSHTM MSc Research Ethics Committee and the Ethical Committee of the Federal Medical Centre of Makurdi. Permission was also granted by the Benue State Director of Public Health and Director of Health Planning. (See Appendix d. for letters of confirmation and LSHTM CARE form.)
9
Candidate no. 107467
3. Results
3.1. The local context: Mental health services in Benue State
As of the 2006 census, the population of Benue State was above 4.25 million, with about a three percent annual growth rate (National Population Commission 2010). The land area of the state is over 31,000 km2, divided into 23 local government areas (LGAs) (National Population Commission 2010). As in other low-resource settings in Nigeria, the burden of mental and neurological disorders that is expected among a population this size is not adequately addressed by existing services (WHO and MOH 2006).
At the tertiary level, the population is served by a Department of Psychiatry at the Federal Medical Centre in Makurdi. A Department of Psychiatry is also being set up within the Benue State University Teaching Hospital, although the department is still lacking essential staff and equipment. Primary care is mainly the responsibility of LGA governments, but prior to signing agreements with the Benue State CCMHP, they made little provision for patients with mental and neurological disorders.
For nearly a decade, the gap in community-level mental health services has been partially filled by three charity-run CBR facilities. Two of these facilities, Edawu and Agboke, currently operate in partnership with CCMHP, and are described further below. The third facility was established by NKST, a predominately Tiv Christian Reformed Church (Rubingh 2002), in the town of Mkar in central Benue State. NKST has yet to formalize its relationship with CCMHP.
3.1.1. Private community-based rehabilitation facilities
Both CBR facilities offer outpatient clinics, including outreach to less accessible LGAs. Due to the high unmet need for mental health services, these facilities are overwhelmed with additional patients from Benue and neighbouring states. The lone CPN at Agboke claims he has served 10,000 new clients since its opening in 2004. Edawu, which was established in 1996 and is allegedly more popular than Agboke, reported 323 new clients last quarter alone, although its information system has historically been too weak to provide any estimate of total clients served.
Amidst overwhelming demand, with many patients travelling great distances despite seasonal and infrastructural barriers, these clinics have adopted a dispensary role, mostly treating psychosis and epilepsy. Psychosocial interventions are time-intensive and treatment is not yet standardized due to delays in the release of Nigeria’s mhGAP guidelines. Quality of care, however, is better at the main outpatient clinics, which operate three times per week, than at their outreach clinics, which are served on a monthly or bi-monthly basis and thus have much higher patient loads
10
Candidate no. 107467 coupled with weaker infrastructure. CPNs at CBR facilities regularly see 30-60 clients on a given outreach day, and 20-40 or more at their main clinics. A consultation generally lasts 5-10 minutes, and often less during outreach. Transport within Benue State can be arduous, particularly during the rainy season, and not all outreach facilities provide sheltered waiting space. Asked about typical wait times at clinics, one patient responded that she sometimes sits for several hours on a bench outside of the clinic, adding, “It’s hard.” Sympathetic staff do their best to deliver rapid service, often at the expense of more comprehensive care.
The need for clinical supervision is acute, as CCMHP reported in 2012: “Inconsistent supervision of clinics of the rehabilitation centre almost threatens the quality of care administered to patients. This is because the state government has yet to second... the Project Clinical Officer” (CCMHP 2012: 16). After over two years of delays, the secondment of a Clinical Officer to CCMHP still has not been effected.
CBR facilities receive donor funding from CBM and other charitable organizations to whom they are accountable. The facilities’ existing M&E systems are not robust enough to facilitate reliable reporting. Donor reports of process data are prepared quarterly through painstaking procedures of tallying information from daily registers, while thousands of patient files accumulate data in the form of long-hand clinical notes—arduous to prepare and virtually impossible to analyse. Meanwhile, concerns about the accuracy of reporting limit its utility to both supervisors and donors.
Figure 2. CBR donor reports show evidence of calculation errors.
11
Candidate no. 107467
3.1.2. Public clinics in primary health centres CCMHP anticipates that scaling up mental health service coverage by establishing a public mental health clinic in each LGA will result in a dramatic re-distribution of patient loads, eliminating the need for concurrent CBR outreach and ultimately leading to better quality of care at CBR clinics. The process has been slow, however, and the opening of new clinics is far behind schedule.
Unlike CBR facilities, the new mental health clinics are being established within PHCs through public-private partnerships. While CCMHP provides for CPN training and offers an initial seed grant of drugs and other—mostly in-kind—support, the LGA is largely responsible for the day- to-day funding and management of the clinic. Tedious bureaucratic processes cause severe delays, especially in securing financial commitments from LGAs. For example, as part of the Memorandum of Understanding (MOU) with CCMHP, each LGA is expected to provide petty cash on a monthly basis for recurrent operational expenses. However, these disbursements are unreliable. Clinics that do not receive their disbursements on time have no budget for printing, file folders, or other supplies necessary for record-keeping. Further, the MOU does not provide for filing cabinets or for additional staff to aid the CPN in record-keeping and other clinical duties.
In spite of these challenges, the first PHC clinic, Agasha, opened in Guma LGA in July 2013. Only two patients were served over the first week of operations. The CPN spends several days per week conducting awareness-raising activities in the community in order to identify cases and improve clinic attendance, but uptake has been lower than expected. The CPN attributes this to seasonal market fluctuations: “The feeling I got from the community leaders was that I’d find many [cases]… but then, this June-July is never the best for the pocket… it makes the pocket shallow, because now these farmers have sold all their crops, they are waiting for new ones [to grow].” Other CPNs have suggested they only expect two to three patients per day once their clinics open. It remains uncertain whether patient loads are likely to shift from CBR Figure 3. A new CPN is provided a desk and chairs for facilities to new clinics. his PHC clinic, but no place to store files.
12
Candidate no. 107467
Table 3. Summary of key information on clinics
CBR Clinics PHC Clinics Agboke Edawu Agasha History & context Established 2004 1996 2013 Founders Benue State Methodist Benue State Methodist CCMHP, Diocese of Otukpo Diocese of Igede Guma LGA Location (LGA) Ohimini Oju Guma Organisation Public/Private Private Public Model Following the Amaudo model, outpatient services are Outpatient mental health provided on-site and at outreach clinics, as an extension clinic operating out of a of the inpatient CBR facility. specialized PHC. Funding and management Funding is provided by CBM and the Methodist Diocese, Local government is with smaller in-kind contributions from other local and responsible for general international partners. CCMHP offers technical expertise funding and management. and supervision support. CCMHP covers some start- up costs and offers technical expertise and supervision support. Main clinic Frequency 3 days/week 3 days/week Client load ~20-40 clients/day - Clinical characteristics Mostly psychosis, epilepsy - Clinical interventions Mostly older psychotropic medications mhGAP essential medications Psychosocial interventions Limited; some counselling mhGAP psychosocial interventions Outreach clinics Frequency 1-2 days/month per site - Client load ~30-60 clients/day - Clinical characteristics Mostly psychosis, epilepsy - Clinical interventions Mostly older psychotropic medications - Psychosocial interventions Severely limited; little counselling - Clinic staff CPNs 1 1 1 Non-specialist aides 2 2 -
3.2. Feasibility, acceptability, and appropriateness of the M&E system
A M&E system consists of five main stages: (1) data collection; (2) processing; (3) analysis; (4) dissemination; and (5) use (WHO 2005). These results focus on the first two stages, though their implications for further analysis, dissemination, and use are also discussed. Key themes generated from the analysis of case study data are presented below, and are organised by their main impact on the feasibility, acceptability, or appropriateness of the piloted system at each stage in the M&E process. Where possible, these themes are illustrated through direct quotes from CPNs, excerpts from field-notes, and photographs.
13
Candidate no. 107467
3.2.1. Challenges to data collection
Feasibility
Data collection takes place in the context of low human, financial, and material resources at both the existing CBR clinics and newer PHC clinics. These challenges are exacerbated by high demand at CBR clinics and the additional constraints of outreach. High patient loads strain existing infrastructure and procedures—which are already sub-optimal for data collection— and place severe time constraints on CPNs. Although the feasibility of data collection at the new PHC clinics is not yet affected by high patient loads, the lack of clear procedures and very limited resources available during start-up may present future challenges if they are not rectified.
Key themes that emerged during fieldwork concerned the lack of resources and procedures to ensure secure storage and efficient retrieval of patient files, the recurrent expenses incurred by data collection, and the high workload of the CPNs who are responsible for data collection, as described in the following.
Figure 4. Filing systems break down under the strain of high patient loads and poor storage capacity.
14
Candidate no. 107467
Storage and retrieval
During our observation visits at one CBR clinic, clients complained of unusually long wait times while the CPN’s aides attempted to locate patient files. Lack of an appropriate filing system and storage space for the large volume of patient files hampers retrieval, and it was our impression that these procedures are often ignored in order to speed up service delivery, as a CCMHP staff member observed: “Normally they just give them pills and they [clients] go.” When a file could not be located quickly, case notes were written on a blank sheet of copier paper and set aside to be filed later. The daily register, which provides a record of all patient encounters, was then used to look up critical information from previous visits, such as diagnosis and prescription, to guide treatment.
At the other CBR clinic, a more orderly system has been implemented, though lack of secure storage space remains a challenge. The CPN often records the same clinical information on multiple records, including the patient appointment cards that clients are asked to bring to each visit. As the CPN explained, “Anything can happen, this place can burst into fire. So I can [also] take the treatment from here [the appointment card]. This is much safer in the patient’s hand.”
In order to facilitate clinical decision-making, patient data needs to be accessible during the clinical encounter; however, patients may choose to seek services either at outreach or at the main clinic. CPNs travel by motorcycle, and cannot transport large volumes of paper to each site. As one CPN explained, “For mobile outreach, it is a big problem. Because we have 600 records [from one outreach site], we’re only bringing the ones for new patients.” Follow-up patients who are well- known to this CPN are then treated based on information recorded in the daily register or by memory. Even for the new patients, the process of retrieving files during outreach remains inefficient, and often the correct file cannot be located.
The other CPN currently offering outreach has addressed this issue by arranging for files to be stored either at the main clinic or at outreach sites, based on client preference. Contact persons at the outreach sites are willing to communicate with the CPN by phone to access and update patient files remotely. Figure 5. Many outreach sites have no storage for files.
15
Candidate no. 107467
However, the reliability and confidentiality of this system are of concern, and this option is not available to all CPNs. The new CPN at Agasha, for example, has been invited to start outreach clinics in other PHCs around his LGA, but he cannot keep files at these centres. He explains, “I’m concerned about the security of the tools in those places.” Even his main clinic has no Figure 6. Paper records incur significant damage. shelves or other storage space. The CPN has resorted to keeping tools in an unlocked desk drawer in his clinic.
Recurrent expenses Data collection incurs direct costs proportional to the patient load at each clinic. At CBR clinics, each patient is issued an individual appointment card and patient file, as well as either copier paper or more structured tools for recording clinical notes—all of which are subject to loss or damage, and must often be replaced. In their current formats, the enrolment form and six-month review forms are six pages each, and a routine follow-up form is one page. If the new system is implemented with no change to the tools, a patient who attends clinic on a monthly basis may accumulate 18 times more paperwork during their first year than they normally would at a CBR clinic.
For PHC clinics, recurrent expenses are particularly problematic, due to the issues in petty cash flow described earlier. At Agasha, the CPN has been operating with no office supplies for weeks, due to delays in the release of LGA funds.
Workload
As will be discussed further in the next section on the acceptability of data collection, CPNs at CBR clinics struggle with very high workloads. They are often aided by several non-specialist health workers, who help with file retrieval and administrative data collection. Provisions for additional staff have not been made at the new PHC clinics, however. Although the CPN at Agasha was able to manage alone during his first two weeks of services, he expressed concern that this might not be feasible in the long-run, as recorded in field-notes from a clinic visit:
16
Candidate no. 107467
The supervisor asks him if he thinks he can fill out these tools, appointment cards, and also dispense the medications and talk to patients. He says yes, ‘but I might need some help.’ He says that the only people around who can help are ‘students’, but they ‘have their limitations,’ given their relative lack of training and experience.
CPNs too lack training and experience in terms of record-keeping, and there are currently no standard operating procedures to guide either the CPN or any other health centre staff who might be willing to aid them in their services.
Figure 7. Non-specialist staff aid record-keeping at CBR facilities.
17
Candidate no. 107467
Acceptability
Patient loads heavily affect the time constraints placed on CPNs, especially during outreach, though CPNs from both the CBR clinics and the new PHC clinics complained of the length of the tools, as described further below.
Tool length One of the CPNs from the CBR clinics was unwilling to use the tools during outreach, due to high patient loads. The other agreed to test the enrolment form at outreach, but the first trial-run did not go smoothly, as he later recalled: “I was to see 60-something patients, so when I had that new case, I had to abandon these other patients to complete the form… the other patients were complaining, they went home late because of it. It took a long time.” Not only did the tool take “a long time”—over 40 minutes, according to our observations— it also appeared to interfere with the patient-provider interaction, as recorded in field-notes from the consultation:
While using the form... he seems to be interacting with her less fluidly, staring at the form, asking a question, then going back to the form, etc...The patients after her are obviously impatient, getting irritated about the wait—I can hear them smacking their lips with disapproval.
The trial-run thus raised concerns about the acceptability of the tools to patients and families, who may endure longer waits to receive what may also appear to be less empathetic care.
The new CPN at the Agasha PHC clinic was able to deliver the enrolment form at his main clinic without compromising the quality of services, as recorded in field-notes:
Overall, he does a very thorough job using the tools, and manages to balance his writing and discussion with the caregiver... He complains that there isn’t enough space for all he wants to write for the medical history and physical and mental state examination, and he turns the page over to write on the reverse side.
The exchange took about 50 minutes, due to his diligent note-taking. He was also more enthusiastic about using the tools than many of his peers, explaining, “It helps me because there’s a guide there, and all I have to do is tick some boxes and fill in a few things, so it’s like a standard, it’s helpful.” In the absence of the high patient loads observed at CBR clinics—especially during outreach— the new CPNs may be more willing to commit time to data collection, and may even embrace the tools as service aids to ensure a high standard of care.
All six CPNs interviewed were generally willing to use the tools at their main clinics, although they expressed ongoing reservations about the length of enrolment and six-month review forms. As
18
Candidate no. 107467 one new CPN put it, “It’s very, very long... too bulky, but no problem, we’ll do it.” Another estimated it will take him approximately an hour to complete an enrolment form, when he’d prefer to spend no more than 30-45 minutes per new patient. However, the CPNs agreed that with training and practice, they will be able to administer the tools with more speed and fluidity. Prior experience from the pilot suggests that the time to complete an enrolment or six-form could be reduced to a minimum of 20-40 minutes.
Appropriateness
CCMHP was founded in 2011, and is still at an early stage of implementation. It remains unclear what services will be available to patients in Benue State. Thus, the tools were not adequately adapted to reflect the local context before being distributed to CPNs and used for training. Further, CPNs received inadequate training in the use of the tools, and do not receive sufficient clinical supervision. As a result, they are currently unable to complete many questions validly and reliably, especially with respect to clinical outcomes.
Service availability All six CPNs identified issues in the adaptation of the rehabilitation plan sections of M&E tools. Options for pharmacological treatment, referrals and psychosocial interventions do not reflect the range of services actually provided to clients.
Issues with CCMHP’s drug supplier, for example, resulted in one clinic operating with no supply of benzhexol. Although the CPN would like to offer it to control the psychomotor side effects of some anti-psychotics, the drug simply is not available. The decision, then, of how to complete a “medications prescribed” field is imbued with a recurrent tension between best practice and actual practice. Prescribed treatment may be quite different from the treatment available at the clinic.
One CPN explained that he had been skipping the referrals section, since many of the services listed are either unavailable or inaccessible to most clients. The trainer advised him to complete this section on referrals “as prescribed”, rather than as an actual record of referral. However, this strategy would undermine the validity of referral data and lead to an underestimation of the effect size of referral on patient outcomes in further analysis.
Similar challenges arose in the reporting of psychosocial interventions, as mhGAP guidelines for psychosocial interventions have not yet been finalised for Nigeria, and the training materials did not define the content of clinical interventions. One CPN, for example, often advises family members to include the patient in household chores. During walk-throughs of the enrollment tool and in actual practice, this advice was classified as “daily living skills training”, “family
19
Candidate no. 107467 education/support”, and/or “parenting skills training”. Indicators must be defined in order to be valid and reliable, and to avoid underestimating effect sizes.
Training and supervision CPNs had difficulty calculating outcome measures and symptom severity scores using most standardised scales. They attributed this to poor training. Participants complained that the half-day M&E training was “too fast,” as we heard from one CPN: “I’m finding it a bit difficult… I even said they should give us one week [of training], but you know there’s money involved, because of accommodation and feeding. Nothing is difficult if you have time [to learn it].”
CPNs and other staff who attended the training said that the Global Assessment of Functioning (GAF) score—a key indicator of patient outcomes—was particularly challenging, and requires additional instruction. One CPN said that the training caused “great confusion,” even though a standard GAF scoring chart was provided. Another explained in further detail:
We had an exercise and the scenario was such that it was difficult to distinguish [between] those [GAF scores] that were very close. Trying to establish where one range ends and another begins is hard—how do you know you are now beyond 25 and into 30? What if you want to say, ‘between 25 and 30?’
During walk-throughs of the tools and observations of data collection at the CBR clinics, CPNs also had difficulty using the two disorder-specific scales, the Clinical Global Impression for Schizophrenia (CGI-SCH) scale and the Patient Health Questionnaire (PHQ-9) for depression. During walk-throughs of the PHQ-9, one CPN said it was “not easy,” and “hard to do immediately,” but believed it would become easier with further training: “By studying it we’ll become used to it, no problem.”
The six CPNs all preferred using the Clinical Global Impression Severity (CGI-S) scale to any other outcome measure. CGI-S rates the severity of the patient’s condition based on the CPN’s clinical experience, using a relatively simple one to seven scale, for any mental or neurological disorder.
3.2.2. Challenges to data processing
Feasibility
The main challenge to data processing is workload, as described further below. The current human resources cannot accommodate the additional burden of routine data processing using the piloted M&E system.
20
Candidate no. 107467
Workload CPNs do not have reliable computer access, adequate training, or sufficient time to enter data from their own patient files into a database. Adding data entry to the terms of reference of the CCMHP Project Coordinator, Program Manager, or Project Clinical Officer position has been suggested as an alternative. However, the sheer volume of data that will be generated state-wide may be more than a single staff member can regularly process.
With around 300 new patients per quarter per facility, CBR clinics could potentially produce over 14,000 pages of enrolment data to be entered each year. This does not include the six-month follow-up data to be entered for all enrolled patients.
The opening of new clinics will be staggered over the next two years or longer, and patient loads should not threaten data entry initially, while these clinics are still raising awareness about their services. In the long-term, however, it will be difficult for one person to enter data from a growing patient population at 23 different LGA sites around the state, as files are not to be removed from clinics, and intrastate transport is often a challenge.
The question of who should enter the data remains unresolved, as the Project Coordinator and Program Manager already have very high workloads, and neither has been formally trained in M&E or database management. Further, CCMHP endeavours to build a sustainable program that can be transferred fully to state and local government management at the end of its funding cycle. By this logic, the Clinical Officer should be made responsible for data entry. However, many state Clinical Officers are close to retirement age, and CCMHP worries that they could be assigned an officer who does not have the computer literacy necessary to manage an Access database, as one staff member explained: “If we get someone young we can send them to trainings, but some of these older people from the government, they aren’t even trained to use a computer.” If the officer does retire on the job, any training inputs are wasted. Further, the officer will already have a great deal of work simply carrying out clinical supervision—including supervision of data collection— and may not be able to prioritise data entry.
Acceptability
Data entry can be arduous, and as one supervisor explained, only certain individuals have the “personality”,“analytical mind”, and “attention to detail” required. CCMHP staff fear that they may be gambling on both the capacity and willingness of the Project Clinical Officer to see that the clinics’ data are accurately processed.
21
Candidate no. 107467
Appropriateness
As noted in Appendix a., the Access database used for data entry at Amaudo and Abuja has some design flaws. Missing “null” response options may yield missing data. The forms contain some string fields that are laborious to enter, and ultimately unnecessary for analysis. The original forms in Access also no longer match the revised tools, and will need to be updated.
3.2.3. Challenges to data analysis, dissemination, and use
As the system has not yet been implemented in Benue State, a thorough treatment of the feasibility, acceptability, and appropriateness of these stages in the M&E process cannot be made here. However, it is worth noting that “upstream” issues in data collection and processing preclude any further manipulation of the data “downstream.”
Obviously, if data collection and processing are not simultaneously feasible, then routine data will not be generated for analysis. If bottlenecks are limited to the processing stage, then the tools themselves can still be used on a case-by-case basis to judge service quality, and may provide a useful resource for supervision and even non-routine evaluation. Monitoring, however, will not be possible at the program level, and neither will subsequent dissemination of routine data to donors and other stakeholders.
Dissemination via the MHIS reporting system presents a particular challenge. Although it would not be challenging to generate a MHIS report from the Access database, monthly reporting would be impossible using the pilot system, which only requires input of enrolment and six-month follow-up data. Monthly indicators, such as the demographics of patients seen in the previous month, would require use of the register and additional processing.
22
Candidate no. 107467
4. Discussion
It is clear from the results of this case study that use of the piloted M&E system in its current state will not be feasible, acceptable, or appropriate without significant changes to the M&E tools and processes, as well as additional resource inputs. Many of the challenges identified—especially at the stages of data collection and processing—are common to LMIC settings, as previous research on MHIS implementation has demonstrated (Ndetei and Jenkins 2009). Researchers have noted that many of these issues remain unresolved even in high-income countries (Ndetei and Jenkins 2009). They call for further research into solutions for LMIC settings, in order to provide evidence for “the most realistic practices” (Ndetei and Jenkins 2009:15). This discussion considers the implications of the local context for the design of better tools and processes for the M&E and MHIS, respectively, in order to suggest realistic solutions for Benue State.
4.1. Implications for M&E system
The comprehensive design of the M&E system is attractive to many stakeholders, as it aims to address a variety of information needs on a routine basis. However, this design is ultimately one of the system’s biggest flaws. The M&E tools serve three different purposes, acting as: (1) clinical records used by CPNs in daily practice; (2) administrative records gathering process data for supervision and reporting purposes; and (3) research instruments capable of generating outcome data for quality improvement and evaluation. When data needs are not adequately prioritised, the tools become too “bulky” for routine data collection and processing in environments where high workloads and severe resource constraints cause frequent implementation failure.
To make the M&E system feasible and acceptable, changes should be made both to its design and to the resources allocated. In recognition of the varying patient loads and infrastructural capacity among clinics, less time-intensive procedures should be implemented in outreach versus main clinics, and in older CBR clinics versus newer PHC clinics. Less-experienced CPNs may benefit from the use of M&E tools, which provide “a standard” to follow during the clinical encounter. Further, these tools can be more easily audited for clinical supervision than the unstructured clinical notes written by CPNs at the older CBR clinics. However, more experienced CPNs operating under severe constraints may not be able to meet this “standard”, and the challenges of mobile outreach are prohibitive.
Even for PHC clinics, the M&E tools should be simplified, prioritising the needs of CPNs, then supervisors, donors, and researchers, in that order. CGI-S could be used as a primary outcome measure, as it was widely accepted by participating CPNs. The decision of whether to retain GAF as a
23
Candidate no. 107467 secondary outcome measure should be contingent upon CPNs’ comfort with the scoring tool following further training. The added benefit of including disorder-specific tools, such as the PHQ-9 and the CGI-SCH, is perhaps not worth the costs in terms of time, further training and supervision, and the potential of generating unreliable, invalid, or missing data. Separate service aides could easily be developed to assist CPNs in the identification of depression and schizophrenia, which might also be more user-friendly, culturally appropriate, and less time-consuming to administer.
The challenges of printing, transporting, storing, and processing paper-based tools suggest that mobile technology would be a worthwhile investment. A mobile platform could automate much of the analysis and dissemination of data for use by a variety of stakeholders, supporting the more sophisticated M&E for which this pilot system was originally designed. Larger, short-term donor investments in infrastructure and training for durable technological solutions may in fact be easier to secure than long-term government funding for recurrent costs like office supplies and staffing, given the recent experiences of CCMHP.
4.2. Implications for MHIS
Results of the case study suggest that any routine data collection and processing requires simple tools that do not add considerably to CPNs’ workload. However, implementing the MHIS and M&E systems in their current formats will double the data collection responsibilities placed on CPNs. Further, according to the Benue State M&E Officer, the services of lower-level M&E officers who currently process LGA data for HIS reporting will not be available to mental health clinics until the MHIS is scaled up nationally. The CPN will therefore be responsible for processing data from the daily register for the monthly MHIS report. The current reporting tool requires time-consuming disaggregation of indicators by age and gender that are carried out by hand using paper tools (see Appendix c. for suggested revisions).
Even if bottlenecks in processing data from the pilot M&E system were eliminated, for example by hiring a data clerk, the system would be unable to produce monthly MHIS data. This means a daily MHIS register must be kept, even if the indicators used by the M&E system and the MHIS are harmonised. However, CPNs at CBR facilities have aides who assist them with record- keeping. Where possible, the task of entering data from M&E tools into the daily register could be shifted to these less-specialised staff.
Given the amount of work required of the CPN simply to fulfil MHIS reporting requirements, this system should not be used in tandem with the M&E system at high patient-load CBR clinics. As will be explored in the next section, if the register is revised to include a minimum dataset necessary
24
Candidate no. 107467 for clinical decision-making by the CPN, the MHIS could be used as the routine system across all clinics, including outreach, while the piloted data collection tools could be used at PHC clinics for further non-routine processing and analysis. (Figure 8. provides a diagram of the proposed routine non-routine data pathways.) If mobile technology were introduced, however, the daily register could be eliminated, as the HIS report could be generated from data collected on a daily basis using a smart-phone or tablet.
25
Candidate no. 107467
5. Recommendations
Table 4. condenses the results of this study into eight main challenges—four for the M&E system and four for the MHIS, respectively. Key recommendations are made to address each of these challenges in the context of a revised system integrating elements of the MHIS and M&E systems, as described briefly in the following sections. This section concludes with an outline of next steps for scaling up M&E in Benue State, and recommendations for further research.
Table 4. Key challenges and recommendations for MHIS and M&E based on findings Challenges Recommendations M&E Tools 1. Tools are “bulky” and time-consuming to a. Limit the number of response options to multiple-choice questions. administer. b. Limit response options in rehabilitation plan to available treatments and services. 2. Clinical outcome scales are time- c. Remove disorder-specific scales (PHQ-9, CGI-SCH). consuming and difficult to use for CPNs d. Retain CGI-S as primary clinical outcome indicator for all disorders. (see 1.) e. Consider retaining GAF as secondary clinical outcome indicator for all disorders, following further training. M&E Processes 3. Routine data collection using M&E tools f. Clarify routine and non-routine data pathways; carry out routine is too time-consuming to be carried out data collection using MHIS tools at outreach and other high-load at outreach or other clinics with high sites. client loads (see 2.) g. Provide standard operating procedures and storage materials to improve efficiency of file retrieval during data collection. 4. Routine data processing cannot be h. Use CCMHP staff only for non-routine data entry (see f.) carried out using existing human i. Consider mobile technology to eliminate the need for storage, resources. retrieval, and processing of paper-based tools (see g.) MHIS tools 5. Design of MHIS tools is not user-friendly. j. Redesign register with separate sections for enrolment and follow- up, to reduce the amount of duplicate data recorded for each client over the course of treatment (see i.) 6. MHIS tools do not use validated outcome k. Harmonize MHIS and M&E tools (see d., e.) scales. MHIS processes 7. Collecting data with MHIS and M&E tools l. Enable non-specialized CPN aides to copy data from M&E tools into simultaneously increases burden on CPN. daily register (see i.,k.)
8. Clinic M&E officer will not be available to m. Redesign register to enable CPN to tally data easily for reporting process data from MHIS register for (see i.) reporting until MHIS is scaled-up n. Redesign the report to reduce the amount of aggregation/ nationally. disaggregation of data required by CPN (see i.)
25
Candidate no. 107467
5.1. Changes to processes The key output of this research is a revised system design integrating the MHIS and M&E system (see Figure 8).The MHIS is used primarily for routine monitoring across all community mental health clinics and outreach in Benue State, and replaces the current CBR donor reporting process. The M&E system is reserved for additional, non-routine evaluation at newer PHC clinics.
Although the MHIS register must be comprehensive enough to enable clinical decision- making and reporting in the absence of M&E tools, the M&E tools are retained at newer clinics. Supervisors are expected to audit these tools randomly and observe data collection during site visits to ensure data quality. For the purposes of evaluation, a random or judgment sample of tools is taken from these PHC clinics for processing by the Clinical Officer or another CCMHP staff member on an as-needed basis. The tools also provide data for more formal research, on the condition that researchers provide short-term staffing for any large-scale data entry. These changes are all expected to be accompanied by detailed standard operating procedures (SOPs) and training materials, which will be designed following approval of the system design by CCMHP stakeholders. SOPs will highlight options for task-shifting responsibilities to less specialized staff.
5.2. Changes to tools Indicators used by the MHIS and M&E system are harmonised, such that data may be copied by hand from M&E tools into the register outside the context of the clinical encounter, preferably by non-specialist staff. Minor changes are made to MHIS and M&E tools to address redundancies and issues in the logic of specific questions. Rehabilitation plan options are adapted to reflect the treatment and services currently available. Further tool-specific changes are described below.
5.2.1. Changes to M&E tools PHQ-9 and CGI-SCH are removed from the M&E tools. CGI-S is retained as the primary outcome measure. GAF is retained; however, further training is recommended, as is a post-training analysis of the reliability and validity of GAF scores calculated by CPNs in a service delivery context.
5.2.2. Changes to MHIS tools The design of the MHIS register and report are simplified to facilitate more accurate and efficient pen-and-paper processing by CPNs. The register is divided into an enrolment section and a follow-up section, so that data may be aggregated separately for each of these two patient categories using a simple tallying process. Further, the indicators used by the report are modified, requiring less complicated disaggregation of data. CGI-S is adopted as the primary clinical outcome. Meaningful indicators cannot be calculated for change in GAF score using a pen-and-paper tallying method. (See Appendix c. for revised MHIS tools.)
26
Candidate no. 107467
Figure 8. Recommended system integrating MHIS and M&E pilot tools into routine and non-routine data pathways
27
Candidate no. 107467
5.3. Next steps
The next steps before scaling-up the proposed new system are: (1) to finalise tools and design of the new system in consultation with stakeholders; (2) to develop SOPs and training materials for CPNs and supervisors; (3) to carry out the training, conducting a pre- and post-test to measure improvement in reliability and validity of GAF scoring; (4) to pilot the new system at the existing CCMHP-supported clinics; and (5) to conduct a process evaluation of the pilot, ideally using a lot-quality assessment (Valadez et al 1996) or similar methodology to assess data quality. Pending favourable results, the system can then be considered for state-wide scale-up.
An additional output of this project will be a grant proposal to fund further research on mobile technology. A randomised control trial with an economic evaluation component could test the pen-and-paper version of this system against one using mobile devices. The results would provide much-needed evidence about innovations to improve the feasibility of implementing robust M&E systems for community mental health in LMIC.
28
Candidate no. 107467
6. Conclusion
In LMIC, community mental health programmes are beset by severe resource constraints in environments of acute need. This situation calls for better evidence to fuel innovation and sharing of best practices among LMIC, as well as arguments for further investment. However, as demonstrated by this research, resource constraints also impede routine M&E, which might otherwise provide much-needed evidence. This formative research, and the pilot that preceded it, take a design approach to help break the cycle, asking how M&E tools and processes can take into account the limitations of the local context. The main output is an integrated system design that links a bare- bones routine M&E pathway—carried out primarily via the MHIS—to a non-routine pathway enabling enhanced M&E when and where this is feasible. However, not enough has been done in global mental health to address the need for resources, in turn, to facilitate M&E. Short-term investments, for example in mobile technology, could ultimately improve the long-term sustainability of M&E and provide robust arguments for scaling up community mental health programmes in low-resource settings. WHO’s (2001) calls for better community mental health monitoring and researchers’ dismay over the lack of evaluation data from LMIC must be met with a commitment not only to research and development for better MHIS and M&E system designs, but also to better training, supervision, and infrastructure to support these systems.
29
Candidate no. 107467
7. References
Academy of Medical Sciences (2008) Symposium Report: Challenges and priorities for global mental health research in low- and middle-income countries. Academy of Medical Sciences: London.
Bowling A. (2009) Research Methods in Health: Investigating Health and Health Services. Open University Press: Baltimore.
CCMHP (2012). CCMHP Biannual Report: January-June 2012. Comprehensive Community Mental Health Programme Headquarters: Otukpo, Benue State.
Cohen A., Eaton J. Radtke B., George C., Manuel B.V., De Silva M., Patel V. (2011) ‘Three models of community mental health services in low-income countries’, International Journal of Mental Health Systems [online], vol. 5, no. 3, p.11. Available from:
Cohen A., Eaton J., Radtke B., De Menil V., Chatterjee S., De Silva M., Patel V. (2012) ‘Case study methodology to monitor & evaluate community mental health programs in low-income countries’. Available from:
Colwill R. (1993). ‘Psychiatric nursing: Amaudo, the village of peace’, Nursing RSA, vol. 8, no. 1, January, pp. 23-32.
Eaton J. and Agomoh A.O. (2008). ‘Developing mental health services in Nigeria’, Social Psychiatry and Psychiatric Epidemiology, vol. 43, no. 7: 552-558.
Eaton J., McCay L., Semrau M., Chatterjee S., Baingana F., Araya R., Ntula C., Thornicroft G., Saxena S. (2011) ‘Scale up of services for mental health in low-income and middle-income countries’, Lancet, vol. 378, no. 9802, October, pp. 1592-1603.
Hanlon C., Wondimagegn D. and Alem A. (2010) ‘Lessons learned in developing community mental health care in Africa’, World Psychiatry, vol. 9, no. 3, October, pp. 185-189.
Jenkins R. (2003). ‘Supporting governments to adopt mental health policies’, World Psychiatry, vol. 2, no. 1, February, pp. 14-19.
Jenkins R., Baingana F., Ahmad R., McDaid D., Atun R. (2011). ‘Health system challenges and solutions to improving mental health outcomes’. Mental Health in Family Medicine, vol. 8, pp. 119- 127.
Kusek J.Z. and Rist R.C. (2004) Ten Steps to a Results-Based Monitoring and Evaluation System. World Bank: Washinton, D.C.
Lafond A. and Fields R. (2003) ‘The Prism: Introducing and analytical framework for understanding performance of routine health information systems in developing countries’. USAID. Available from:
National Population Commission (2010) ‘Population distribution by age and sex’, Census Priority Table Series, vol. 4, April. Available from: < http://www.population.gov.ng/index.php/publications /141-population-distribution-by-age-and-sex-2006-census-priority-tables-vol-4>. [22 August 2013].
30
Candidate no. 107467
Ndetei D.M. and Jenkins R. (2009) ‘The implementation of mental health information systems in developing countries: Challenges and opportunities’, Epidemiologia e Psichiatria Sociale, vol. 18, no. 1, pp. 12-16.
Rubingh E.F. (2002) ‘Saai, Josua Elisa Isholibo, 1911 to 1987: Nongu U Kristu u Ken Sudan Hen Tiv (Church of Christ in the Sudan Among the Tive), Nigeria’, Dictionary of African Christian Biography [Online], Available from:
Szlezák N.A., Bloom B.R., Jamison D.T., Keusch G.T., Michaud C.M., Moon S., Clark W.C. (2010) ‘The Global Health System: Actors, Norms, and Expectations in Transition’, PLoS Medicine [online], vol. 7, no. 1, January, p.4. Available from:
Valadez J.J., Brown L.D., Vargas W.V., Morley D. (1996) ‘Using Lot Quality Assurance Sampling to Assess Measurements for Growth Monitoring in a Developing Country’s Primary Health Care System’, International Journal of Epidemiology, vol. 25, no. 2, pp. 381-387.
WHO (2001) ‘Mental health: New understanding, new hope’, World Health Report. World Health Organisation: Geneva.
WHO (2003) ‘Organisation of services for mental health’, Mental health policy and service guidance package. World Health Organisation: Geneva.
WHO (2005) ‘Mental health information systems’, Mental health policy and service guidance package. World Health Organisation: Geneva.
WHO (2006) ‘Mental health monitoring and evaluation’, Mental health policy and service guidance package, World Health Organisation: Geneva.
WHO (2010) Mental Health Gap Action Programme Intervention Guide. World Health Organisation: Geneva.
WHO and MOH (2006) WHO-AIMS Report on Mental Health System in Nigeria. World Health Organisation and Nigeria Ministry of Health: Ibadan.
WHO (2010) Health information systems. World Health Organisation: Geneva.
31
Candidate no. 107467
Appendix a. Contextual challenges encountered during pilot
Data collection At the Abuja pilot site, a single CPN is responsible for a large and diverse catchment area of over 8,000 km2 and 1.4 million residents, serviced largely through home visits conducted by car (Cohen et al. 2011). According to a CBM supervisor involved in the pilot, detailed information could not be collected from patients who did not speak either English or the same local language as the CPN, due to a lack of appropriate translation. The same supervisor indicated that the CPN had difficulty locating patients at six- month intervals for follow-up, resulting in a relative dearth of outcome data for analysis. In contrast, the Amaudo pilot was carried out by a CPN serving a catchment area of only 245 square km2 and 125,000 residents (Eaton et al. forthcoming). The CPN is responsible for mental health services at five PHCs, but the majority of his clinic-based work takes place at one site (Eaton et al. forthcoming). He also conducts weekly outreach via motorcycle (Eaton et al. forthcoming). Although more incomplete data was received from Abuja, conditions during mobile outreach are not amenable to data collection at either site, according to the supervisor. As a result, both CPNs often chose to record data after the clinical encounter, or even after returning from outreach, having already served 20-30 patients in the field. Poor recall may therefore have compromised data quality and contributed to the large quantity of missing data noted by analysts, as the supervisor explained: “If you don’t fill it [out] the first time, you often forget to come back to it [the tool] later.”
Data processing Incentives were paid to data entry personnel; however, the CBM supervisor responsible for data entry at Amaudo noted that the process was tedious, and significant attention may not have been paid to ensuring data quality either during entry or at supervision visits to CPNs. Turnaround was severely delayed due to competing demands on data entry personnel, and LSHTM analysts waited roughly 20 months to receive data from 127 patients total. The Access database used for data entry also has design flaws, such as missing “null” options for some questions; in these cases, non-applicable responses cannot be distinguished from missing data.
Data analysis, dissemination, and use Outcome data could not be analysed at the pilot sites due to poor data quality (personal communication)—likely a result of challenges in both data entry and processing, as well as inadequate mechanisms for supportive supervision and problem-solving to identify and rectify these challenges in the early stages of the pilot. Long lag-times between data collection and transmission to analysts also impeded timely dissemination and use of data for routine monitoring. Thus the utility of the system for both evaluation and monitoring purposes was limited at pilot sites.
I
Candidate no. 107467
Appendix b. Piloted M&E tools tested during formative research
Appendix b.1. Enrolment form...... III
Appendix b.2. Routine follow-up form...... X
Appendix b.3. Periodic review form...... XI
II
Enrolment Form Patient ID
COMPLETE THIS FORM IF THE PATIENT WHEN A PATIENT IS ENROLLED IN THE PROGRAMME A. ADMINISTRATIVE INFORMATION
1. Patient ID P P P 0 0 0 0
2. Date of registration with Programme D D / M M / Y Y Y Y
3. New or existing client? New Existing
4. Has patient given his/her consent for his/her data to be stored on the database? Yes No
5. First name(s) Surname
6. State LGA
7. Community
8. Contact person, relationship to patient Contact Telephone number
9. Psychiatric Volunteer Nurse/clinician (write ‘none’ if not applicable)
B. DEMOGRAPHIC INFORMATION
10. Date of Birth (or nearest approximation) D D / M M / Y Y Y Y
11. Age Adult (19 yrs or older) Child (younger than 19yrs)
12. Gender Male Female
13. Marital status Never married Married Co-habiting
Separated Widowed Divorced
14. Number of children
15. Education level None/ Not completed primary school Completed primary school Completed secondary school
Further education Literacy classes/literate
16. Current occupation Pre-school Student/training Manual worker
Non-manual worker Keeping house/homemaker Subsistence (e.g.farming, petty trading)
Retired Unemployed Professional
Specify exact current occupation
Comprehensive Community Mental Health Programme for Benue State M&E system III
Candidate no. 107467 C. INFORMATION ABOUT THE PROBLEM
17. Number of years patient has had problem Y Y M M (only enter months if patient has had problem for less than 1 year, and write ‘00’ in years box)
18. Problem category (put a number in all the boxes that a patient has a problem with, starting with 1 for the most important problem, 2 for the second most important etc .) Mental illness (psychosocially impaired) Low vision Blind
Physically impaired Intellectually impaired Hearing impaired
Epilepsy Not impaired
Other If ‘Other’, specify here:
D. DIAGNOSIS AND SEVERITY OF CONDITION
19. Medical history. Please make notes on the history of the patient’s problem, for example the background to the problem, for epilepsy, how often the seizures occur and the type of epilepsy.
IV
Enrolment Form Patient ID
Physical and Mental state examination
20. Mental health diagnosis Put a number in all the boxes that in your clinical judgment, the patient is suffering from, starting with 1 for the most important problem, 2 for the second most important etc. Please also enter the exact diagnosis in this box, for example “drug induced psychosis” or “schizophrenia”. For alcohol/substance abuse, please state which substance in the last box. Depression Epilepsy Psychosis/manic depression
Dementia Alcohol/ Substance abuse Anxiety Disorder
Child mental disorder Other (if ‘other’, specify exact diagnosis below)
Specify exact diagnosis/ substance used
Comprehensive Community Mental Health Programme for Benue State M&E system V
Candidate no. 107467 21. Clinical Global Impression Severity (CGI-S): Considering your total clinical experience with this particular population, how mentally ill is the patient at this time? (complete for all patients)
Normal, not at Borderline Mildly ill Moderately ill Markedly ill Severely ill Among the most all ill mentally ill extremely ill 1 2 3 4 5 6 7
22. PHQ-9: Screening for depression* ONLY complete for ADULTS, 19 and older. DO NOT complete for children 18 and below.
Over the last 2 weeks, how often have you been bothered by any of Several More than Nearly every the following problems? (circle the number in the cell to indicate your Not at all days 7 days day answer)
1. Little interest or pleasure in doing things 0 1 2 3
2. Feeling down, depressed, or hopeless 0 1 2 3
3. Feeling tired or having little energy 0 1 2 3
ONLY complete the following questions 4 - 9 if the patient answers yes (ie. grey boxes) to two or more of the above questions.
Over the last 2 weeks, how often have you been bothered by any of Several More than Nearly every Not at all the following problems? days 7 days day
4. Trouble falling or staying asleep, or sleeping too much 0 1 2 3
5. Poor appetite or overeating 0 1 2 3
6. Feeling bad about yourself—or that you are a failure or have let 0 1 2 3 yourself or your family down 7. Trouble concentrating on things, such as reading the newspaper or 0 1 2 3 watching television 8. Moving or speaking so slowly that other people could have noticed. Or the opposite—being so fidgety or restless that you have been 0 1 2 3 moving around a lot more than usual 9. Thoughts that you would be better off dead, or of hurting yourself 0 1 2 3 in some way
Add numbers in columns (questions 1 to 9 inclusive): + +
TOTAL:
Interpretation of TOTAL: 0-4 5-9 10-14 15-19 20-27 None Mild depression Moderate Moderately severe Severe depression depression depression
23. ONLY complete the following questions for patients with EPILEPSY (check first box if does not have epilepsy) Patient does not have epilepsy
How many seizures has the patient had in the past month? (write ‘00’ if none)
24. ONLY complete the following questions for patients with PSYCHOSIS (check first box if patient does not have psychosis) Patient does not have psychosis/schizophrenia
VI
Enrolment Form Patient ID
Clinical Global Impression for Schizophrenia (CGI-SCH): Severity of illness
Considering your total clinical experience with
patients with psychosis, how severely ill is the
not ill Minimally ill Mildlyill Moderatel yill Markedly ill Severelyill Amongthe most severelyill patient? Normal,
POSITIVE Symptoms 1 2 3 4 5 6 7 (e.g. hallucinations, delusions, bizarre behaviour)
NEGATIVE Symptoms 1 2 3 4 5 6 7 (e.g. affective flattening, poor motivation or anhedonia)
DEPRESSIVE Symptoms 1 2 3 4 5 6 7 (e.g. sadness, depressed mood or hopelessness)
COGNITIVE Symptoms 1 2 3 4 5 6 7 (e.g. impaired attention, concentration or memory)
OVERALL SEVERITY 1 2 3 4 5 6 7
E. FUNCTIONING AND QUALITY OF LIFE
25. Global Assessment of Function scale (Refer to the GAF SCORING SHEET for CHILDREN OR ADULTS as appropriate) Enter a score between 0 and 100 based in your clinical judgement of the patient. GAF score
26. Record any comments you have on the patient’s quality of life
F. FAMILY ATTITUDES TOWARDS PROBLEM
27. Attitude of family towards problem Positive Accepting Ashamed (try to change attitude of (open about problem) (try to hide problem) others)
28. Willingness of family to help patient Supportive and want to help Reluctant to help Refuse to help
Comprehensive Community Mental Health Programme for Benue State M&E system VII
Candidate no. 107467 G. REHABILITATION PLAN
29. Medication prescribed (use generic names) None (skip to "Interventions administered" question below) Anti-psychotics Anti-epileptics Anti-depressants Other
Chlorpromazine Carbamazepine Amitriptyline Diazepam/Nitrazepam Haloperidol Phenobarbital Fluoxetine Benzhexol Fluphenazine Decanoate Trifluoperazine
30. Medication frequency and dosage Name of medication Dose Route Frequency No. days prescribed Medication 1 Tablets (Oral)
Injection (Parenteral)
Name of medication Dose Route Frequency No. days prescribed Medication 2 Tablets (Oral)
Injection (Parenteral)
Name of medication Dose Route Frequency No. days prescribed Medication 3 Tablets (Oral)
Injection (Parenteral)
31. Total cost of medications prescribed per month (if no payment made, record as ‘00’) NGN
32. Interventions administered/prescribed by Fieldworker or Mental Health Supervisor (tick all that apply or None “None”) Family education/support Daily living skills training Counselling (e.g. problem solving therapy)
Economic integration Human rights protection Parenting skills training
Other (specify if ‘other’)
33. Referred for other services (tick all that apply or “None”) None External mental health services Vocational training General medical care
Self-help/peer support groups Orthopaedic/physiotherapy Ear care
Early intervention for childhood disabilities Employment/micro-finance Eye care
Educational support/return to school Legal Other (specify)
VIII
Enrolment Form Patient ID
H. NOTES/ANY OTHER RELEVANT INFORMATION (Please continue on a separate sheet if necessary)
Comprehensive Community Mental Health Programme for Benue State M&E system IX
Routine Follow-up Form Patient ID
USE THIS FORM FOR ANY ROUTINE FOLLOW UP VISITS EXCEPT 6 MONTH FOLLOW UP WHEN SPECIFIC ‘PERIODIC REVIEW FORM’ SHOULD BE USED.
First name Family Name
Date today Date of enrolment interview
Progress since last review Social and Psychological: Any changes to social circumstances or issues highlighted by patient or family? Any change in mental state?
Medication: Current medication(s) including dose regimen (list ALL current medications) Is patient willing to take medication? Any problems obtaining medication? Any side effects?
Other: (Any physical or other issues not mentioned above?)
Plan until next review:
Signed: (Name and position)
Comprehensive Community Mental Health Programme for Benue State M&E system X Periodic Review Form Patient ID
COMPLETE THIS FORM EVERY SIX MONTHS AFTER A PATIENT IS ENROLLED IN THE PROGRAMME 1. ADMINISTRATIVE INFORMATION
1. Patient ID
2. Age Adult (19 yrs or older) Child (younger than 19yrs.)
3. Time in months since enrolled in clinic (this review form should be used every 6 months) 6 months 12 months 18 months 24 months Other (specify) months
Date of periodic review (d/m/year) / /
4. Number of patient contacts with Community Psychiatric Nurse in past 6 months
2. TREATMENTS PRESCRIBED IN THE PAST 6 MONTHS
5. Has there been a change in the type of medication prescribed in the past 6 months? Yes No
6. If yes, why was the change made? Side effects Patient choice Ineffective
Cost Other If ‘Other’ please specify reason:
7. Over the past 6 months, how often has the patient/family paid for their medication? All of the time Most of the time Some of the time None of the time
8. Psycho-social interventions attended in the last 6 months (tick all that apply or “None”) None Family education/support Daily living skills training Counselling (e.g. problem solving therapy)
Economic integration Human rights protection Parenting skills training
Orientation and mobility Communication/sign Other (specify) language
9. Referred for other services in the last 6 months (tick all that apply or “None”) None External mental health services General medical care Self-help/peer support groups
Early intervention Orthopaedic/physio Educational support
Employment/micro-finance Vocational training Eye care
Ear care Legal Other (specify)
3. MEDICATION AND SIDE EFFECTS
10. In the past month, how often has the patient been taking the medicine prescribed by the programme? All of the time (all days, every dosage) More than half the time
Less than half the time None of the time
Comprehensive Community Mental Health Programme for Benue State M&E system XI
11. If patient didn’t take their medication all of the time, why not? Patient taking medication as prescribed
Patient decision Missed appointment
Cost Medication not available
Other If ‘Other’ please specify:
12. Side effects of prescribed medication None Does not significantly interfere with patient functioning
Significantly interferes with patient functioning Side effects outweighs therapeutic effect
Record any comment you have on side effects the patient is experiencing
4. CLINICAL OUTCOME ASSESSMENT
13. Clinical Global Impression Severity (CGI-S): Considering your total clinical experience with this particular population, how mentally ill is the patient at this time? (complete for all patients) Normal, not at Borderline Mildly ill Moderately ill Markedly ill Severely ill Among the most all ill mentally ill extremely ill 1 2 3 4 5 6 7
PHQ-9: Symptoms of depression* ONLY complete for ADULTS. DO NOT complete for children.
14. Over the last 2 weeks, how often have you been bothered by any Several More than Nearly of the following problems? (circle the number in the cell to indicate your Not at all days 7 days every day answer)
1. Little interest or pleasure in doing things 0 1 2 3
2. Feeling down, depressed, or hopeless 0 1 2 3
3. Feeling tired or having little energy 0 1 2 3
ONLY complete the following questions 4 - 9 if the patient answers yes (ie. grey boxes) to two or more of the above questions.
Over the last 2 weeks, how often have you been bothered by any of Several More than Nearly Not at all the following problems? (use “✓” to indicate your answer) days 7 days every day
4. Trouble falling or staying asleep, or sleeping too much 0 1 2 3
5. Poor appetite or overeating 0 1 2 3
Comprehensive Community Mental Health Programme for Benue State M&E system XII Periodic Review Form Patient ID
More than Over the last 2 weeks, how often have you been bothered by any of Nearly every Not at all Several days half the following problems? day the days 6. Feeling bad about yourself—or that you are a failure or have let 0 1 2 3 yourself or your family down 7. Trouble concentrating on things, such as reading the newspaper or 0 1 2 3 watching television 8. Moving or speaking so slowly that other people could have noticed. Or the opposite—being so fidgety or restless that you have been 0 1 2 3 moving around a lot more than usual 9. Thoughts that you would be better off dead, or of hurting yourself 0 1 2 3 in some way
Add numbers in columns (questions 1 to 9 inclusive): + +
TOTAL:
Interpretation of TOTAL: 0-4 5-9 10-14 15-19 20-27 None Mild depression Moderate Moderate- severe Severe depression depression depression
15. ONLY complete the following questions for patients with EPILEPSY (check first box if does not have epilepsy)
Patient does not have epilepsy How many seizures has the patient had in the past month? (write ‘00’ if none)
15. ONLY complete the following questions for patients with PSYCHOSIS
Patient does not have psychosis/schizophrenia
Clinical Global Impression for Schizophrenia (CGI-SCH): Severity of illness
Considering your total clinical experience with
patients with psychosis, how severely ill is the
patient?
Normal, not ill Minimally ill Mildlyill Moderatel yill Markedly ill Severelyill Amongthe most severelyill
POSITIVE Symptoms 1 2 3 4 5 6 7 (e.g. hallucinations, delusions, bizarre behaviour)
NEGATIVE Symptoms 1 2 3 4 5 6 7 (e.g. affective flattening, poor motivation or anhedonia)
DEPRESSIVE Symptoms 1 2 3 4 5 6 7 (e.g. sadness, depressed mood or hopelessness)
COGNITIVE Symptoms 1 2 3 4 5 6 7 (e.g. impaired attention, concentration or memory)
OVERALL SEVERITY 1 2 3 4 5 6 7
Comprehensive Community Mental Health Programme for Benue State M&E system XIII
5. FUNCTIONING
16. Global Assessment Functioning Scale (Refer to the GAF SCORING SHEET for CHILDREN OR ADULTS as appropriate) Enter a score between 0 and 100 based in your clinical judgement of the patient.
GAF score 17. Record any comments you have about change in the patient’s quality of life
6. SOCIAL OUTCOMES
18. Attitude of family towards problem Positive Accepting Ashamed (try to change attitude of family) (open about problem) (tried to hide problem)
19. Willingness of family to help patient Supportive and want to help Reluctant to help Refuse to help
20. Have any of the following things changed in the patients’ life in the past 6 months? If yes, how have they changed? Changed No Changed for better change for worse Comment Marital status/relationship
Education/training
Job/occupation
Treatment by others (discrimination)
Other (specify)
7. REHABILITATION PLAN
21. Are you changing the rehabilitation plan as a result of this 6 month review? Yes No
22. Medication prescribed (use generic names) None (skip to "interventions administered" question) Anti-psychotics Anti-epileptics Anti-depressants Other
Chlorpromazine Carbamazepine Amitriptyline Diazepam
Haloperidol Phenobarbital Fluoxetine Nitrazepam
Trifluoperazine Benzhexol
Fluphenazine Decanoate
Comprehensive Community Mental Health Programme for Benue State M&E system XIV Periodic Review Form Patient ID
Medication frequency and dosage
Name of medication Dose Route Frequency No. days prescribed Medication 1 Tablets (Oral)
Injection (Parenteral)
Name of medication Dose Route Frequency No. days prescribed Medication 2 Tablets (Oral)
Injection (Parenteral)
Name of medication Dose Route Frequency No. days prescribed Medication 3 Tablets (Oral)
Injection (Parenteral)
23. Total cost of medications prescribed per month (if no payment made, record as ‘00’) NGN
24. Interventions prescribed by Community Psychiatric nurse/clinician (tick all that apply or “None”) None Family education/support Daily living skills training Counselling (e.g. problem solving therapy)
Economic integration Human rights protection Parenting skills training
Other (specify if ‘other’)
25. Referred for other services, CBR or other (tick all that apply or “None”) None External mental health services General medical care Self-help/peer support groups
Early intervention Orthopaedic/physio Educational support/return to school
Employment/micro-finance Vocational training Eye care
Ear care Legal Other (specify)
8. NOTES/ANY OTHER RELEVANT INFORMATION (Please continue overleaf if necessary)
Comprehensive Community Mental Health Programme for Benue State M&E system XV
Appendix c. Revised MHIS Tools
Appendix c.1. Revised MHIS daily register: Enrolment register...... XVII
Appendix c.2. Revised MHIS daily register: Follow-up register...... XVII
Appendix c.3. Revised MHIS monthly report...... XVIII
XVI
MHIS Daily Register Section 1: Enrolment clients (part 1) Sex and age profile Diagnosis Referrals
Males only Females only
Name Client’s
Client Caregiver’s No Date of village/ Contact no.
ID name isorder
client community
15) 15) 20) 20) 25) 25) 30) 30) 35) 35) 40) 40) 45) 45) 50) 50) 15) 15) 20) 20) 25) 25) 30) 30) 35) 35) 40) 40) 45) 45) 50) 50)
------
5) 5) 10) 10) 5) 5) 10) 10)
------
(0 (0 (6 (6 (11 (11 (16 (16 (21 (21 (26 (26 (31 (31 (36 (36 (41 (41 (46 (46 (51+) (51+) (0 (0 (6 (6 (11 (11 (16 (16 (21 (21 (26 (26 (31 (31 (36 (36 (41 (41 (46 (46 (51+) (51+) Depression/anxiety Depression/anxiety Epilepsy Epilepsy depression depression Psychosis/manic Psychosis/manic Dementia Dementia abuseabuse Alcohol/subtance Alcohol/subtance disorder d mental mental Child Child referral referral Incoming Incoming referral referral Outgoing Outgoing referral referral No No 1 2 3 4 5
MHIS Daily Register Section 2: Follow up-up clients (part 1) Visit Sex and age profile Diagnosis Referrals
type Males only Females only
Client’s
Name Caregiver’s
up up
Client - village/ Contact no. No Date of name ID community (if changed) client (if changed)
(if changed)
follow
15) 20) 25) 30) 35) 40) 45) 50) 15) 20) 25) 30) 35) 40) 45) 50)
d mental disorder mental d
------
visit of the month of the visit
5) 10) 5) 10)
- - - -
st
1 Additional (0 (6 (11 (16 (21 (26 (31 (36 (41 (46 (51+) (0 (6 (11 (16 (21 (26 (31 (36 (41 (46 (51+) Depression/anxiety Epilepsy depression Psychosis/manic Dementia abuse Alcohol/subtance Chil referral Incoming referral Outgoing referral No 1 2 3 4 5
XVII
MHIS Daily Register Section 1: Enrolment clients (part 1)
Risk Treatment Drugs administered Dosage CGI-S score
Additional Comments
Others Others Notes
Normal, not at all ill all at not Normal, ill mentally Borderline ill Mildly ill Moderately ill Markedly ill Severely most the Among ill severely
help
-
Evidence of persistent risk to to risk persistent of Evidence others or self to risk potential of Evidence others or self or self to risk of evidence No others treatment Pharmacological Counselling support education/ Family protection Humanrights training skills Parenting Self training skills living Daily Amitryptiline Benzhexol Carbamazepine Chlorpromazine Nitrazepam Diazepam/ Fluoxetine decanoate Fluphenazine Haloperidol Phenobarbital (1) (2) (3) (4) (5) (6) (7)
MHIS Daily Register Section 2: Follow up-up clients (part 2)
Risk Treatment Drugs administered Dosage CGI-S score
sk to self or or self to sk
Additional Comments
Others Others Notes
Normal, not at all ill all at not Normal, ill mentally Borderline ill Mildly ill Moderately ill Markedly ill Severely most the Among ill severely
help
-
luoxetine
Evidence of persistent risk to to risk persistent of Evidence others or self to risk potential of Evidence others or self ri of evidence No others treatment Pharmacological Counselling support education/ Family protection Humanrights training skills Parenting Self training skills living Daily Amitryptiline Benzhexol Carbamazepine Chlorpromazine Nitrazepam Diazepam/ F decanoate Fluphenazine Haloperidol Phenobarbital (1) (2) (3) (4) (5) (6) (7)
XVIII
MHIS Monthly Report
0-5 6-10 11-15 16-20 21-25 26-30 31-35 36-40 41-45 46-50 51+ Client demographics M F M F M F M F M F M F M F M F M F M F M F No. new clients served, disaggregated by age and sex No. follow-up patients served, disaggregated by age and sex
Normal, not at all ill Borderline mentally Among the most Client severity Mildly ill (3) Moderately ill (4) Markedly ill (5) Severely ill (6) (1) ill (2) severely ill (7) No. new clients served, disaggregated by CGI-s score No. follow-up patients served, disaggregated by CGI-s score
No. clients served No. clients served No. clients served No. clients served No. clients served No. clients served based on Depression/ based on Epilepsy based on Psychosis/ based on Dementia based on Alcohol/ based on Child Client diagnosis Anxiety Manic Depression/ Substance Abuse Mental Disorder Bipolar Disorder No. new clients served, disaggregated by diagnosis No. follow-up patients served, disaggregated by diagnosis
Client referrals Number of clients referred into the facility
Number of clients referred out of the facility
XIX
Appendix d. Permissions
Appendix d.1. Ethical approval: Benue State Director of Health Planning...... XXI
Appendix d.2. Ethical approval: Federal Medical Centre Ethics Committee...... XXII
Appendix d.3. Ethical approval: Benue State Director of Public Health...... XXIII
Appendix d.4. LSHTM CARE form application...... XXIV
XX
d.1. Ethical approval: Benue State Director of Health Planning
XXI
d.2. Ethical approval: Federal Medical Centre Ethics Committee
XXII
d.3. Ethical approval: Benue State Director of Public Health
XXIII
Combined Academic, Risk assessment and Ethics (CARE) approval form for MSc Project Reports
*This form must be completed electronically. Please refer to your course’s Project Handbook for guidance. Double-click tick-boxes and select ‘checked’ to put an X in the relevant box.
SECTION 1 – STUDENT AND COURSE INFORMATION MSc DETAILS AND DEADLINES (deadlines to be communicated by Course Director) Academic Year 2012-13 MSc course (and stream, where applicable) Global Mental Health (GMH) Deadline for Supervisor approval Wednesday 03 April 2013 Deadline for Course Director approval Thursday 04 April 2013 Deadline for submission to Ethics Committee Friday 05 April 2013 Target for approved form to be passed to TSO Friday 10 May 2013 STUDENT AND SUPERVISOR DETAILS (to be completed by student) Full name of student Grace Kathryn Ryan Student email address [email protected] Year of study (part-time students First Year Second Year only) Supervisor name Mary DeSilva and Alex Cohen (co-supervisors) Supervisor email address [email protected] and [email protected] Supervisor LSHTM institution/organisation Supervisor status (at time of this Confirmed Provisional Still to be version of the form being completed) identified Name of personal tutor (where Tatiana Salisbury Supervisor is still to be identified)
SECTION 2 – APPROVAL AND SUBMISSION STATUS *Students please note: It is a requirement of your LSHTM degree that you obtain all required approvals before beginning your project work. Your Supervisor and Course Director must specifically give Risk Assessment approval. Ethics approval must also be obtained where necessary (answers in Section 5 will help determine if this is required or not).
STUDENT DECLARATION (to be completed for all projects) I agree to conduct my project on the basis set out in this form, and to consult staff (initially, my Supervisor) if making any subsequent changes – especially any that would affect the information given with respect to ethics approval.
XXIV
I agree to comply with the relevant safety requirements, and will submit a separate request for LSHTM travel insurance where relevant. *Where seeking ethics approval for a study involving human subjects, please also attach copies of any information sheets, consent forms, and other relevant documents. Date of declaration 5 April 2013
Please save the electronic file of this CARE form in the format
[MSc title]_[Year of Submission]_[Surname]_[Forename]_CARE
You will also be required to submit a copy of this CARE form with your final written-up project. This should be anonymised by removing your name and e-mail address from Section 1.
XXV
STAFF APPROVAL *Staff please note: Sections 3 and 4 of the form should be completed by the student before you give approval. Rather than ‘sign’ this form, you should email the student and explicitly confirm approval, e.g. stating “In my role as supervisor, I approve the attached form”. The student is then responsible for updating the form and passing it on to any other staff. However if you would answer ‘no’ to any of the ‘Yes/No’ questions below, or disagree with any of the statements given, or have any other concerns, then you should not give approval. Instead, please contact the student immediately to inform them of your concerns and discuss changes which they may need to make before you may be willing to give approval. Please also be aware that in the exceptional case of a request to undertake a project in a country or region to which the Foreign & Commonwealth Office advise against travel, the student would need to fill out a separate form which will then need further School-level approval by the Safety Manager and Secretary & Registrar.
SUPERVISOR’S APPROVAL (required for all projects – this approval should be given first) Supervisor has agreed that Section 3 of this form is a reasonable Yes No summary of the proposed project. Supervisor has agreed that responses in Section 4 of this form Yes No address the main risks connected with a project of this nature. Supervisor has agreed that responses in Section 5 of this form Yes No correctly indicate whether or not ethics approval will be required. Name of Supervisor (if not yet identified, personal Alex Cohen, Mary De Silva tutor or Course Director should approve) Date of approval 5 April 2013, 8 April 2013
COURSE DIRECTOR’S APPROVAL (required for all projects – should follow Supervisor approval) Course Director has agreed that the proposed project’s academic Yes No content, set out at Section 3 of this form, is suitable for this MSc. Course Director has agreed that responses in Section 4 of this form Yes No address the main risks connected with a project of this nature. Name of Course Director (or nominee) Alex Cohen Date of approval 5 April 2013
FACULTY SAFETY SUPERVISOR’S APPROVAL (only required if project involves working with pathogenic organisms, human blood or radiochemicals – should follow Supervisor approval) Faculty Safety Supervisor has agreed that the proposed project, as Yes No set out in this form and particularly Section 4, may proceed. Name of Faculty Safety Supervisor (or nominee) Date of approval
ETHICS APPROVAL (required for all projects involving human subjects or human data, except for public domain data that cannot enable the identification of living people – NB that Supervisor approval must have been received before the application is submitted to the Ethics Committee) The Ethics Committee has approved the project proposal set out on Yes No this form. Date of approval Ethics Committee application number assigned
XXVI
SECTION 3 – APPLICATION FOR ACADEMIC APPROVAL *All students should complete all sub-sections (3.1, 3.2 and 3.3). If particular questions are not applicable to you then please write ‘N/A’.
3.1 PROJECT OUTLINE (should not normally exceed 750 words total) Proposed project title: (should not normally exceed 20 words) Formative Research to Scale Up Clinical Monitoring and Evaluation for Mental Health Services in Benue State, Nigeria Proposed project type: *See course-specific section of Project Handbook for details of project types permitted for each MSc. Be aware that restrictions may apply for individual courses. Formative research Proposed project length: *For almost all students, this will be ‘Standard’. Extended projects are only available for MSc IID; they have a different schedule and allow a slightly greater word count. Standard Extended Background: (about 200 words) *Indicate why this topic is of interest or relevance. *If the project involves work with a specific organisation please give details. *Please give any other details specifically relevant for consideration by the Ethics Committee, e.g. related to purpose. Nigeria is the most populous country in Africa, home to 170 million people, up to 30% of whom may struggle with mental illness in any given year. Despite this enormous burden, Nigeria does not yet have structures in place for the systematic reporting and management of data for patients accessing mental health services.
Through a public-private partnership between the government of Nigeria, the local NGO Amaudo Itumbauzo, the international charity CBM, and LSHTM researchers, a new mental health information system has been developed and piloted in a primary health care setting and a community rehabilitation setting in Abuja and Amaudo. Pilot data are currently being analyzed by LSHTM statisticians.
Project partners wish to adapt the piloted information system for use in 10-20 new mental health clinics, components of a community mental health programme currently under development in Nigeria’s Benue State. Formative research is needed to ensure that the information system is successfully implemented in this new setting.
The purpose of this project is to inform the adaptation and scale-up of the mental health information system to be used in Benue State, through a combination of original qualitative research and review of qualitative and quantitative data collected through the earlier pilot in Abuja and Amaudo.
Hypothesis: (about 30 words, where applicable) N/A Overall aim of project: (about 30 words) The aim of this project is to inform the adaptation and scale-up of a recently piloted mental health information system for use in 10-20 new clinics in Nigeria’s Benue State. Specific objectives of project: (about 70 words) The objectives of this formative research are to qualitatively assess the feasibility, acceptability, and appropriateness of previously piloted information system tools and processes for use in Benue State through (1) interviews with program staff, administrators, and users; (2) observations of care settings; and (3) review of qualitative and quantitative data collected during the original pilot in Abuja and Amaudo. Proposed methods: (about 200 words) *Please summarise methods, and include any relevant details for consideration by the XXVII
Ethics Committee such as numbers of participants and procedures to be performed. As noted above, the qualitative methods to be used in this formative research include in-depth interviews and direct observation to be carried out in clinical settings among program staff, administrators, and users, until saturation is reached. Structured questionnaires will also be provided to program staff and administrators for self-completion prior to commencement of fieldwork. An estimated thirty participants will be recruited via convenience sampling.
Data collection will be conducted by a single researcher and take place in English. A translator may be made available to assist translation of discussion questions into local languages during interviews, as needed. No identifiable information will be collected about individual participants. General topics of discussion will include (but not be limited to): use of existing tools and systems; needs from future tools and systems; existing supervision and reporting structures; and patient-provider interactions.
Qualitative and quantitative findings from the Abuja and Amaudo pilot will be summarized as text for coding and analysis alongside formative research data.
This data, as well as all field-notes and participant responses, will be coded and analyzed by the researcher using Nvivo-9 qualitative research software. As previously described, key themes of acceptability, feasibility, and appropriateness of piloted tools and processes will be explored. Either Patton’s (1997) Utilization-Focused Evaluation Framework or another appropriate model may also be used to further orient analysis and reporting. References: (max 150 words) *List any key references which will shape the project, including for methods to be used. It should not normally be necessary to quote more than 5 references. This project builds off of unpublished work currently underway by Eaton et al. (forthcoming), notably a prior unfinished case study of Amaudo programs and the information system pilot in Abuja and Amaudo.
Health service researchers from Johns Hopkins University have identified gaps in the literature on formative research in terms of formulating appropriate questions, goals, and implementation strategies, as well as selecting methods for data collection and analysis (Gittelsohn et al. 2006, pp.3); they observe that choosing a specific theoretical framework “provides direction to the overall formative research process” (pp.4). A Ultilization-Focused Evaluation (Patton 1997) framework is preliminarily suggested here for its pragmatic emphasis on “intended use by intended users” (Mark 2005, pp.3). Prior work: (only where relevant; max 100 words) *Indicate any previous work you have done related to this project topic, including student work, professional work, or publications. I have several years experience in qualitative research in sub-Saharan Africa. I received hands-on instruction in qualitative methodologies as part of a School for International Training program in Ghana in 2008. In 2009 I conducted ethnographic fieldwork with mental health service users and personnel in Ghana as part of my undergraduate thesis fieldwork. From 2010 to 2012, I served as a qualitative researcher on a community health team in Rwanda, where I was responsible for the design and implementation of qualitative study components of multiple mixed-methods health service research projects. I also helped to co-write a case study of Rwanda’s mental health programs, which was recently accepted for publication.
XXVIII
3.2 FEASIBILITY (about 100 words total – but can write more or write less if appropriate) What could stop this project from succeeding, or prevent you from achieving your objectives? *Please indicate any aspects of your proposed approach which could potentially experience difficulties, e.g. delays with permissions, data collection or storage problems, lack of sufficient comparable information, etc. You may also wish to mention any wider matters which could affect your project, e.g. civil unrest, natural disasters, transport availability. This project could face logistical complications e.g. delays with permissions from local review boards and unavailability of key informants due to workload or other scheduling concerns. Additionally, although Benue State has not been affected by recent unrest, the Foreign and Commonwealth Office does currently advise against travel to other parts of Nigeria. However, this is a discrete project in a conflict-free state with strong supervision and local linkages, and it is being designed in consultation with LSHTM researchers and CBM representatives with prior experience performing data collection in Nigeria. No travel to at-risk regions of Nigeria will be undertaken. What alternative plans do you have in case you encounter any of the potential problems you have identified? Self-completion questionnaires will be sent out in advance of actual fieldwork, to minimize total direct field-time necessary and avoid additional disruption as much as possible. Transport arrangements will be made in advance, preferably with a vehicle and driver temporarily assigned to the project. Travel and health insurance including emergency evacuation coverage will be procured. Two weeks additional buffer-time will be added to four weeks proposed data collection period to allow for unforeseen delays. If conditions in Nigeria become prohibitive for travel, data collection may be scaled back to self-completion questionnaires and interviews with program staff and administrators by phone.
3.3 DATA SOURCES, INTELLECTUAL PROPERTY AND PERMISSIONS If you expect to use existing data, how will you obtain it? *Indicate who holds the data, who specifically you will contact, and by when. Any contact so far, especially anything confirmed in writing, should be mentioned. Qualitative data from an uncompleted case study of the Amaudo organization has already been made available by Julian Eaton of CBM. Quantitative data from the information system pilot is currently being analysed by Sujit Rahod in collaboration with Mary DeSilva at LSHTM. Mary DeSilva has confirmed that this data will be made available for review, and Sujit Rahod will be contacted in April 2013 regarding data access. If you expect to use any public domain data, please give further details. *Make clear who owns the data and how you will gain access (giving a link if possible). Public domain data must be available to any member of the public, without any restrictions or requirement for special permission, and must not enable the identification of living people. N/A Will any specific data rights permissions or usage limitations be required Yes regarding data to be used or collected in the project? No If ‘Yes’, please describe further. *Remember that local ethics or research governance requirements (see Section 5.2) may entail specific data rights limitations. According to prior communications with Dr. Julian Eaton, there should not be any data ownership concerns of note. Will any copyright agreements or intellectual property rights (IPR) agreements be required regarding data to be used or collected in the project? *Please tick all boxes that apply, and attach copies of any forms/agreements (even if in draft). No specific IPR, copyright or permissions issues should apply to this project (student retains XXIX
copyright and a claim to related IPR) IPR to be retained by LSHTM (specific LSHTM form to be completed) Copyright to be transferred to LSHTM (specific LSHTM form to be completed) IPR, copyright or other agreements/permissions required with external parties/organisations Please give any further relevant details about IPR, copyright or other permissions. N/A
SECTION 4 – APPLICATION FOR RISK ASSESSMENT APPROVAL *All students should answer all questions in sub-section 4.1; this will make clear which of the subsequent sub-sections you need to complete. Ensuring safety during project work is the responsibility of each individual student, and not of LSHTM or LSHTM staff. *Please see the Project Handbook for further guidance.
4.1 TYPE OF RISK (to be completed by all students) Where will the project be carried out? (please tick all that apply) *Note that work away from LSHTM or outside the UK means any form of work for your project, not just primary data collection. Some courses may have specific restrictions on this. All work will take place either at LSHTM, in libraries in the UK, or at my personal residence in the UK. [If so, you do not need to complete either section 4.2 or section 4.3] Some work will take place in the UK that is away from LSHTM sites in London, is non-Library-based, and is not at my personal residence. [If so, section 4.2 on ‘Work away from LSHTM’ must be completed] Some work will take place at my personal residence outside the UK [If so, section 4.3 on ‘Work outside the UK’ must be completed] Some work will take place outside the UK that is not at my personal residence [If so, both sections 4.2 and 4.3 on ‘Work away from LSHTM’ and ‘Work outside the UK’ must be completed] Will the project involve working with or handling any of the following materials? Pathogenic organisms Yes No Human blood Yes No Radiochemicals Yes No [If ‘Yes’ to any of the above, Sections 4.4 and 4.5 must be completed] Are any other potentially hazardous activities likely to be carried out during the project? Yes No [If ‘Yes’, Section 4.5 must be completed] Do any special requirements (e.g. disability-related issues) or other concerns need to be taken into account for either you as a student, study participants or colleagues? Yes No [If ‘Yes’, Section 4.6 must be completed]
4.2 WORK AWAY FROM LSHTM (to be completed if any work will be done away from LSHTM, other than at your home or at libraries elsewhere in the UK) Will the project be based in an established hospital, college, research Yes institute, NGO headquarters, field station or other institutional site? If ‘Yes’, No please give the name and location of the site(s); describe approximately what XXX
proportions of your project will be spent there; and state name and role of person who has confirmed willingness to support you at each site (indicating extent of correspondence, especially what they have confirmed in writing). Julian Eaton of CBM International has confirmed in writing his willingness to serve as an external co-supervisor; he will facilitate contact with local technical advisors (such as his on- the-ground counterpart Emeka Nwefoh) to aid in project supervision and coordination. Although the fieldwork schedule remains tentative, four to six weeks of data collection have been proposed at Benue state clinical sites. Between 10 and 20 clinics in Benue state should be operational by June 2013. Will you have an ‘external supervisor’, co-supervisor or other main advisor, Yes or be working with any specific organisation(s), during your work away from No LSHTM? If ‘Yes’, please indicate the name, role, contact details, and level of support that any such external advisors are expected to provide, and give details about any organisations you will be working with. As previously mentioned, Dr. Julian Eaton of the international charity CBM has offered his support as an external co-supervisor. CBM works extensively with the local organization Amaudo Itumbauzo, which will also help to facilitate fieldwork. Will the project involve personal visits, interviews or interactions with Yes people in their homes, workplaces, community settings or similar? If ‘Yes’, No please give details, including approximately what proportion of your project this will involve. Most of the project data collection will take place among staff members and service users at mental health clinics in largely rural settings. Will the project involve lone/isolated work or significant travel? If ‘Yes’, Yes please give details, including approximately what proportion of your project this will No involve, and state how you can be contacted while working or travelling. As previously noted, much of the data collection will be undertaken independently (i.e. not as part of a formal research team) in rural Nigeria, though the involvement and support of local NGO and clinical staff will be integral to all research activities. Use of a trusted driver and vehicle for research activities has been recommended for logistical and security reasons. I will be in contact via cell phone and email with local research stakeholders and international supervisors as much as possible. What arrangements are proposed for contact with your main supervisor while you are away from LSHTM? Indicate expected ease and frequency of contact, and communication methods to be used. Although internet access at some sites is weak, contact via cell phone, Skype, and email will be feasible on a weekly basis. Please tick to confirm: I have read the LSHTM Code of Practice on off-site work.
4.3 WORK OUTSIDE THE UK (to be completed if any work will be done outside the UK) What form of project work will be undertaken outside the UK? (please tick all that apply) Work at my family home or personal residence only Work at an established hospital, college, research institute, NGO headquarters, field station or other institutional site Work away from my personal residence or an established site *Note that for either the second or third options, you should also have completed Section 4.2. Name the country/countries and region(s) in which work will be undertaken: Country or countries: Nigeria Region(s): Benue State Do the Foreign & Commonwealth Office’s (FCO) Travel Advice Notices Yes (www.fco.gov.uk/en/travelling-and-living-overseas/travel-advice-by-country) advise No against travel to the regions(s), country or countries involved? *Note that if ‘Yes’, the School will not normally permit such travel for project work. In XXXI exceptional circumstances only, requests may be considered by the Safety Committee and require approval by the Safety Manager and Secretary & Registrar. Please tick to confirm: I will seek specific travel health advice before any international travel as part of my project. *Travel health advice, anti-malarials, vaccinations and medication are available from the Hospital for Tropical Diseases Travel Clinic and will be paid for by the School if you follow the procedure set out in the project handbook (TSO need to sign off a form for you). Please tick to confirm: I understand that travel insurance is required when travelling internationally for project purposes. *Free LSHTM travel insurance can be applied for using a separate form, provided the travel is for location-specific project purposes.
4.4 WORK WITH HAZARDOUS SUBSTANCES (to be completed if the project involves any work with pathogenic organisms, human blood or radiochemicals – NB that this will require approval by the Faculty Safety Supervisor) Name the organism or organisms to be used: N/A Identify all potential routes of infection: N/A Name the radiochemical or radiochemicals to be used: N/A List laboratories where work with pathogens or radioisotopes will be carried out: N/A List disinfectants to be used, and describe arrangements for disposal of used material: N/A Will or might Health Surveillance be required for you or any staff working Yes with you? If ‘Yes’, please give details. No N/A
4.5 PRECAUTIONS AGAINST HAZARDS (to be completed if any potentially hazardous activities are likely to be carried out during the project. Refer to Project Handbook and School safety documentation for further information. Faculty Safety Supervisor’s approval may be further requested where felt appropriate by project Supervisor.) Indicate any procedures, activities or aspects of the proposed project which may entail hazards (including work with hazardous substances as per Section 4.4, or anything else relevant). Please set distinct hazards out separately, in a numbered list. N/A Indicate the precautions you will take to prevent or mitigate such potential hazards. Please number these to refer to the specific hazards identified in the preceding question. N/A
4.6 SPECIAL REQUIREMENTS (to be completed if the project involves any special requirements, e.g. disability-related issues, or other concerns that need to be taken into account for either you as a student, study participants or colleagues)
XXXII
What special requirements or concerns need to be taken into account? N/A Do these need to be considered in planning arrangements? Yes If ‘Yes’, please give details. No N/A Do these impact on supervision arrangements? Yes If ‘Yes’, please give details. No N/A Does the project location need to be considered in relation to these? Yes If ‘Yes’, please give details. No N/A Do arrangements for access to specialist medical treatment need to be Yes considered? No If ‘Yes’, please give details. N/A
SECTION 5 – APPLICATION FOR ETHICS APPROVAL *All students should answer all questions in sub-sections 5.1 and 5.2. Answers to 5.1 will make clear whether approval by the LSHTM Ethics Committee is necessary, and which later sub-sections you may need to complete. Section 5.2 covers any external approvals required. *Further detailed guidance about completing this section, and what to do next if formal LSHTM ethics approval is required, is given in Chapter 6 of your Project Handbook. *NB that supervisor approval must be obtained before an application is submitted to the Ethics Committee.
5.1 SCOPE OF STUDY (to be completed by all students) Which of the following applies to your project? (please tick one option only) *Note – the term ‘human data’ includes any documentary data, datasets or biological samples.
Project does not involve any human subjects or any human data. [If so, formal LSHTM ethics approval is not required and you do not need to complete Sections 5.3 or 5.4] Project involves human data, but all this human data is fully in the public domain. [If so, formal LSHTM ethics approval is not required and you do not need to complete Sections 5.3 or 5.4] *Public domain human data must be: available to any member of the public without special permission; to which access is not restricted in any way; and which does not enable the identification of living people, either directly or by linking to other data. Project involves some non-public-domain human data, all of which was previously collected in another study or studies. [If so, formal LSHTM ethics approval is required and Section 5.3 must be completed] Project involves some additional collection of data, further to an ongoing or previously completed study or studies. [If so, formal LSHTM ethics approval is required and Section 5.4 must be completed] Project is a completely new study which will involve human subjects or human data. [If so, formal LSHTM ethics approval is required and Section 5.4 must be completed]
5.2 LOCAL ETHICS APPROVAL OR RESEARCH GOVERNANCE APPROVAL (to be
XXXIII completed by all students) *As well as approval from the LSHTM Ethics Committee, projects may require specific approval from other involved or responsible bodies. For example, in the UK you may need specific authorisation to work in an NHS facility, or to work with vulnerable groups such as patients or children. Outside the UK a wide range of requirements may apply e.g. from local or national Ethics Committees, government departments etc. Students must investigate all potential local approval required for your project work. Failure to check or gain any necessary external approval may invalidate LSHTM approval. Is local approval required for the work being done (whether this approval is Yes still to be obtained, or has already been granted)? No *This should include any forms of ethics approval, research governance approval or other specific permissions that may apply. If ‘Yes’, give details of local approval to be obtained (this must be in place before commencing fieldwork) or which has already been granted. *Please name all bodies whose approval is required, or indicate where work is expected to take place using permissions already granted for a ‘parent’ project. Where approval has already been granted, quote approval reference numbers and if possible give web links to documents. If ‘No’, explain why formal local approval is not required, and describe any less formal permissions, invitations or support you are being given for this work. *If you will be working away from LSHTM with human subjects or human data, but cannot identify a local Ethics Committee or believe that no formal approval is required, then please give details and explain what you have done to check this. In such cases, if you do not have formal approval you should always demonstrate appropriate local support, such as correspondence with local government officials or an involved Non-Governmental Organisation. The Amaudo and Abuja information system pilot was approved as part of the study “Evaluating packages of care for people with severe mental disorders for scaling up in low and middle income countries: a pilot project”, submitted to the LSHTM ethics committee by Vikram Patel and Alex Cohen in 2008. CBM advisors have written that local ethical approval may not be necessary to conduct additional observation and qualitative interviews with no identifying information in Nigeria, and that a formal letter of support from local stakeholders should suffice. However, in the interest of assuring the highest ethical standards for this project, it will additionally be submitted to the Federal Medical Centre in Makurdi, Benue State. *If any specific data rights permissions or usage limitations will be required regarding data to be used or collected in the project (e.g. as a result of local ethics or research governance requirements), this should be spelt out in Section 3.3 earlier.
5.3 PROJECTS USING ONLY PREVIOUSLY-COLLECTED HUMAN DATA (to be completed if project involves non-public-domain human data, datasets or biological samples previously collected in another study or studies; if collecting any new data, complete Section 5.4 instead) Summary of purpose and methods of the original study or studies: (max 100 words) N/A Give details of all approvals under which the original study or studies took place: *Please quote names of Ethics Committees and approval reference numbers (required if previous approval was from LSHTM); if possible give web link to original study application. N/A Proposed study: Ensure that the project outline given in Section 3.1 states the purpose, methods and procedures of the new work to be done in your project, and describes how this builds on the previous study or studies (for which participants will already have been recruited, data or samples collected, and procedures performed). Do not reproduce here. Will your analyses be for purposes not covered by the original application Yes
XXXIV
detailed above? If ‘Yes’, indicate how you will obtain (i) permission to use the data No from the principal investigator responsible for each original study; and (ii) retrospective consent, where appropriate, from the participants in each original study. N/A Does the project involve analysis of documentary information and/or data Yes already collected from or about human subjects? If ‘Yes’, specify analyses No briefly. N/A Does the project involve laboratory analysis of human biological samples Yes already collected, or new or additional analysis of stored samples? If ‘Yes’, No specify the laboratory analyses or tests to be performed. N/A Specify how confidentiality will be maintained. Where data will be anonymised, specify how this will be done. When small numbers are involved, indicate how possible identification of individuals will be avoided. N/A State how your data will be stored and what will be done with it at the end of the study. N/A
5.4 PROJECTS COLLECTING ANY NEW HUMAN DATA (to be completed if project involves collection of human data, datasets or human biological samples – either as a completely new study, or collecting additional data further to an ongoing or previously completed study) Proposed study: Ensure that the project outline given in Section 3.1 contains sufficient detail (inc. purpose, methods, procedures for both new data collection and any work building on previous studies), so as to allow the Ethics Committee to make an informed decision without reference to other documents. Do not reproduce here. Is your project an intervention study? Yes For LSHTM ethics approval purposes, ‘interventional studies’ include all trials based on No random allocation of interventions, and also non-randomised interventions where participants or groups of participants are given treatments (of whatever nature) that they would not otherwise be receiving in the ordinary course of events and which are allocated by the investigators. Will any human biological samples be collected? If ‘Yes’, specify details. Yes No N/A Will any human biological material be stored at LSHTM for more than 24 Yes hours? If ‘Yes’, specify which samples and how and where they will be stored. No *Further guidance is given at http://intra.lshtm.ac.uk/support/research/humantissueact.html N/A Specify the number - with scientific justification for sample size – age, gender, source and method of recruiting subjects for the study. Due to the challenging context in which this fieldwork will be performed, a convenience sample of available clinical and program staff, administrators, and beneficiaries will be interviewed and/or observed until saturation is reached at each field site. Efforts will be made to ensure that this sample includes a range of demographic characteristics (age, gender, etc.) as much as possible. The sample size is estimated at 30, based on projections for the formative research that preceded the original information system pilot, as described by Vikram Patel and Alex Cohen in their 2008 submission to LSHTM for ethical review “Evaluating packages of care for people with severe mental disorders for scaling up in low and middle income countries: a XXXV pilot project.” State the location and likely duration of new or additional human data collection, and the extent to which this will be carried out by you alone, or in collaboration with others, or by others. I will be largely responsible for all new qualitative data collection. I plan to spend four to six weeks collecting data at multiple clinical sites in eastern Nigeria, as specified elsewhere in this document. A translator may be present in order to facilitate interviews with non-English- speaking beneficiaries. State the potential distress, discomfort or hazards, and their likelihood, to which research subjects may be exposed (these may include physical, biological and/or psychological hazards). What precautions are being taken to control and modify these hazards? Potential risks to participants are minimal; however, discussions of prior experiences of mental illness and care may result in patient distress. Any patient contact will take place in a care setting, where adverse events can be addressed through standard procedures. In consent/assent forms and during discussions, participants will be assured that they are not required to participate and may end the discussion at any time without penalization. Specify how confidentiality will be maintained. Where data will be anonymised, specify how this will be done. When small numbers are involved, indicate how possible identification of individuals will be avoided. Patient data has already been anonymized and aggregated before being reported to LSHTM researchers; review of quantitative pilot results will therefore not threaten confidentiality.
During qualitative data collection, all participants will be assigned a subject code and a consent code. Consent/assent forms will be marked with the consent code and kept separate from all other research documents. A password-protected file matching consent codes to subject codes will be kept on a password-protected computer equipped with anti-virus software. The subject code will be marked on the participant’s questionnaire and used to identify any other individual-level data during data coding and analysis. Pseudonyms will be used in reporting results, and identifying details (e.g. village, age, and gender) will be omitted. State how your data will be stored and what will be done with it at the end of the study. All computers containing study data will be password-protected and installed with anti-virus software, and all research documents will be kept in a secure space in a locked room. Any transcripts or other hard copies of research data will be scanned and transferred to password- protected electronic files and then destroyed upon completion of analysis. Electronic files of de- identified qualitative data will be kept for reference by the researcher. State the manner in which consent will be obtained from subjects. Written consent is normally required. Where not possible, explain why and confirm that a record of those giving verbal consent will be kept. Where appropriate, please state if and how the information and consent form will be translated into local language(s). All participants will be provided with written consent forms and a verbal explanation in English by the researcher. Illiterate participants will acknowledge consent with a thumbprint. Caregivers will consent on behalf of vulnerable participants (e.g. cognitively impaired patients), who in turn will be asked for assent. Due to the limitations of project resources, a translator may not be available. In this case, data collection will be limited to English-speaking participants. While this will limit the number of eligible patients for inclusion, the focus of this project is on program staff and administrators, all of whom must provide proof of English language proficiency in order to carry out their work. Please tick to confirm: I have attached copies of the information sheet(s), consent form(s), and other relevant documents related to work with human subjects. As well as collecting new data, will your project also make use of any human Yes data or biological samples collected in a previous study or studies? If ‘Yes’, XXXVI
summarise the purpose and methods of the original study or studies – for which No participants will already have been recruited, data or samples collected, and procedures performed. (max 100 words) The pilot project “Evaluating packages of care for people with severe mental disorders for scaling up in low and middle income countries: a pilot project” informed the development of the mental health information system, which was then piloted and field-tested in Amaudo and Abuja. Quantitative data gathered through this system was de-identified and aggregated for review by LSHTM researchers, and qualitative data was collected using the Case Study Methodology developed Alex Cohen et al. (2012). These results will be summarized as text, in consultation with the investigators, and coded and analyzed alongside data from the original research proposed in this application. Give details of all approvals under which the original study or studies took place: *Please quote names of Ethics Committees and approval reference numbers (required if previous approval was from LSHTM); if possible give web link to original study application. “Evaluating packages of care for people with severe mental disorders for scaling up in low and middle income countries: a pilot project”, submitted to LSHTM Ethics Committee April 2008 by Vikram Patel and Alex Cohen. Will your analyses be for purposes not covered by the original ethics Yes approval detailed above? If ‘Yes’, indicate how you will obtain (i) permission to use No the data from the principal investigator responsible for each original study; and (ii) retrospective consent, where appropriate, from the participants in each original study. No, the results of this study are being used as intended—to “develop benchmarks which will make it possible to begin to compare the effectiveness of programs over time and to compare programs in different settings” (Patel and Cohen 2008). However, I am requesting access to the de-identified data prior to publication. The investigators have already given me permission to review and discuss their results with them, in order to apply lessons learned to the adaptation of the system for Benue State in a timely manner.
XXXVII