Community Survey 17 Health facilities supported by USAID through Project Santé in Centre, , -Est, , Sud, and Grand’Anse departments

May 2019

VRS Central and South Project

This publication was produced at the request of the United States Agency for International Development. It was prepared independently by LOGIK.

Community Survey 17 Health facilities supported by USAID through Project Santé in Centre, Ouest, Sud-Est, Nippes, Sud, and Grand’Anse departments

DISCLAIMER This study/report is made possible by the support of the American People through the United States Agency for International Development (USAID.) The contents of this study/report are the sole responsibility of the VRS Central and South Project and do not necessarily reflect the views of USAID or the United States Government.

i

VRS Central and South Project

Contact Person: Rachelle Cassagnol, MD, MPH Chief Of Party 6, Impasse Hardy, Route du Canapé-Vert, Juvénat, Pétion-Ville, Haïti. Email: [email protected]

The USAID | VRS Central and South Project, is funded by the U.S. Agency for International Development in Haiti (USAID | HAITI) under contract no. AID-521-C-14-00009, beginning August 18, 2014. This contract is implemented by LOGIK

ii

CONTENTS List of tables ...... iv List of figures ...... iv Acronyms ...... v Executive summary ...... vi I. Introduction ...... 1 1.1 Project context ...... 1 1.2 Purpose of the community survey ...... 1 II. Survey Methodology ...... 1 2.1 Sampling...... 1 2.2 Questionnaire...... 3 2.3 Training of the community interviewers ...... 3 2.4 Data collection/ Quality Control ...... 3 2.5 Data analysis ...... 4 III. Results ...... 5 3.1 Survey population ...... 5 3.2 Patients/clients tracking ...... 6 3.3 Patient/client satisfaction at non RBF health facilities ...... 7 3.4 Patient/client satisfaction at RBF health facilities...... 7 3.5 Observation of clients on accessibility of care ...... 8 IV. Main issues and recommendations ...... 9

iii

LIST OF TABLES Table 1 : Number of health facilities assessed by department and level...... 5 Table 2 : Client’s satisfaction score regarding care received between October - December 2018 at non RBF health facilities by items...... 7 Table 3: Client’s satisfaction score regarding care received between October - December 2018 at RBF health facilities ...... 8

LIST OF FIGURES Figure 1: Distribution of respondents by age group ...... 6 Figure 2: Percentage of patients tracked by department ...... 6

iv

ACRONYMS ANC: Antenatal Care

CHW: Community Health Worker

MSPP : Ministère de la Santé Publique et de la Population ODK : Open Data Kit

RBF : Results-Based Financing

USAID: U.S. Agency for International Development

VRS : Vérification des Résultats pour la Santé

WHO : World Health Organization

v

EXECUTIVE SUMMARY The overall purpose of the Vérification des Résultats pour la Santé (VRS) Central and South contract is to support verification of service delivery quantity and quality outputs at sites supported by USAID through Project Santé in six geographic (Centre, Ouest, Sud -Est, Nippes, Sud and Grand’Anse).

The two objectives of this contract are to: 1) Certify that facility-based performance data submitted are reliable. 2) Through periodic household surveys, certify that, at the community level, data related to health care services provided are reliable.

This patient/client survey falls under the second objective of VRS Central and South which is to certify that, at the community level, data related to health care services provided are reliable.

The survey was conducted in March and April 2019 following the quantitative verification of services provided by Project Santé-supported health facilities in the Centre, Ouest, Sud-Est, Nippes, Sud, and Grand’Anse departments during the first quarter of fiscal year 2019.

Purpose and Methodology of the survey The goals of the survey were:  To verify that clients in health facility registers for the first quarter of fiscal year 2019 really exist and received services.  To assess client satisfaction with respect to services rendered at the health facilities and to the management of the health facilities in general among clients who received services during the first quarter of fiscal year 2019.

VRS Central and South data verificators selected a sample of patients/clients from each health facility. Community interviewers used a questionnaire to assess client satisfaction. LOGIK used both electronic and paper-based data collection. For 51 health facilities including all facilities implementing RBF, the community interviewers were trained to conduct interviews and recorded data on the mobile device. The data collected was uploaded to a secure server, aggregated, and extracted in useful format. SurveyCTO which is based on the Open Data Kit (ODK) open source platform was used as a mobile data collection tool.

Key Findings  Out of 4,068 patients/clients selected, 1,827 (45 percent) were found. The tracking rate is 54 percent for facilities implementing RBF and 40 percent for those not implementing RBF.  It was not possible to track 55 percent of the patients/clients selected due mainly to incomplete contact details in the health facility records, no one knows the clients in the neighborhood and the clients live too far from the health facilities.  Among the clients tracked, the general level of satisfaction with health care received between October - December 2018 at health facilities was high as it reached a score of 79 out 100 for both health facilities not implementing the RBF model and implementing the RBF model.

vi

 For both groups, clients were less satisfied with “waiting time”, “opening hours” and “Payments/Fees (Formal and Informal)” compared to the other 4 items evaluated: “Quality of reception, “Perceived quality of care’’, “availability of drugs” and “cleanliness of health facility”

Main issues and Recommendations  Fifty-five percent of the patients selected could not be tracked making it difficult to issue an opinion on the first objective of the survey which is to verify that clients reported in the registers at health facilities really exist in the community and received the services.  The difficulty to conduct spot-check on 55 percent of the patients is a concern especially for facilities implementing RBF. Verification is the basis of a RBF system and the ability to trace patients/clients in the community is crucial. It is important to have registers with detailed clients contacts, addresses and phone numbers.  The overall perceived quality score among patients/clients found is quite high: 79 out of 100. However, the low tracking rate may introduce bias in survey findings because non- respondents may differ from respondents.

vii

I. INTRODUCTION

1.1 PROJECT CONTEXT The overall purpose of the Verification des Résultats pour la Santé (VRS) Central and South contract is to support verification of service delivery quantity and quality outputs at sites supported by USAID through Project Santé in six geographic departments of Haiti (Centre, Ouest, Sud -Est, Nippes, Sud and Grand’Anse).

The two objectives of this contract are to: 1) Certify that facility-based performance data submitted are reliable; 2) Through periodic household surveys, certify that, at the community level, data related to health care services provided are reliable.

This report presents the results of the community survey conducted among a sample of patients/clients who received services during the quarter October - December 2018 at the above- mentioned health facilities. The survey was conducted in March and April 2019.

1.2 PURPOSE OF THE COMMUNITY SURVEY The survey falls under the second objective of VRS Central and South which is to certify that, at the community level, data related to health care services provided are reliable. The goals of the survey were:  To verify that clients in health facility registers for the first quarter of fiscal year 2019 really exist and received services.  To assess client satisfaction with respect to services rendered at the health facilities and to the management of the health facilities in general among clients who received services during the first quarter of fiscal year 2019.

II. SURVEY METHODOLOGY 2.1 SAMPLING During the quantitative verification covering the first quarter of fiscal year 2019 (October – December 2018), VRS Central and South verificators selected a sample of patients/clients who received selected services at the health facilities. As per UC instructions, we did not select patients/clients who received services only at the community level. A sample of 50 patients/clients was selected from each health facility among verified cases in accordance with the following sampling scheme.  For dispensaries, health centers without beds and health centers with beds, the sampling was as follows: o 10 clients were selected from outpatient consultations. o 20 clients were selected from the 4 indicators with the highest bonus per unit. Depending on the services verified in each facility, the 4 indicators filling this

1

criterion may be different. Below is the list of indicators1 ranged from the highest to the smallest bonus:  Number of institutional deliveries.  Number of children fully immunized children by 12 months of age.  Number of bacteriologically confirmed Pulmonary Tuberculosis cases.  Number of pregnant women who made at least a fourth prenatal visit during their last trimester of pregnancy.  Number of cases referred to higher level health facilities.  Number of pregnant women who made their first prenatal visit during their first trimester of pregnancy; Number of pregnant women tested for HIV; HIV positive pregnant women enrolled on ART; Number of HIV positive pregnant women referred to higher level health facilities; Number of TB cases treated; Number of defaulted TB cases re registered; Number of infants born to HIV positive mother placed on ARV prophylaxis within 72 hours.  Number of pregnant women who received at least 2 doses of tetanus toxoid vaccine; Number of acceptors of modern Family Planning methods.  Number of children between 6 and 59 months old who received Vitamin A; Number of children from 6 to 59 months old screened for acute malnutrition at the health facility; Number of children 12-59 months old dewormed.

o 20 patients were selected from the 3 indicators with the highest number of verified cases from the list of indicators mentioned above.

o An additional 10 clients were selected to replace clients that could not be tracked in the original 50 client sample. If a client in the original list of 50 could not be found, the first client in the additional list of 10 was selected to replace him/her; If a second client could not be found he/she was replaced by the second on the additional list and so on.

• For hospitals, the samples of clients were selected from the indicators verified in each facility. A sample of 100 clients/patients was selected in each hospital. If a hospital provided the services described below to less than 100 clients/patients during the quarter, all patients were selected. There are eight indicators for the hospitals: • New outpatient consultations referred by a lower level health facility. • Clients referred by a lower level facility and hospitalized. • Institutional normal deliveries. • Cesarean deliveries. • Counter referral. • Number of TB cases treated • Number of defaulted TB cases re registered • Number of infants born to HIV positive mother placed on ARV prophylaxis within 72 hours.

1 Except the new curative consultations.

2

2.2 QUESTIONNAIRE The survey questionnaire was revised by UC. UC convened a meeting on February 27, 2019 to discuss on the new verification tools and provided the survey questionnaire on February 28, 2019.

The survey questionnaire includes a section on identification of the patient with his name, sex, age, address, phone number, service received, an informed consent statement and 25 questions. The informed consent statement contains the following elements:  The purpose of the survey.  Participation is voluntary, and refusal to participate will involve no loss of benefits to which the subject is otherwise entitled.  The patient may discontinue participation without loss of benefits to which he is otherwise entitled.  The extent to which confidentiality will be maintained.

The questionnaire is divided into 7 sections on client satisfaction: 1. Quality of the reception. 2. Perceived quality of care. 3. Availability of prescribed drugs at the health facility. 4. Client’s opinion on the waiting time. 5. Opening hours. 6. Opinion on Payments/Fees (Formal and Informal). 7. Cleanliness of the health facility.

The questionnaire is presented in Annex A.

2.3 TRAINING OF THE COMMUNITY INTERVIEWERS The survey coordinator trained the community interviewers to conduct data collection. The training included an overview of the VRS project, the purpose of the survey and covered interview techniques, the role of informed consent and confidentiality as well as a thorough review of the questions and the data collection tool. The training consisted of classroom presentations and mock interviews. The community interviewers are people living in each specific community being assessed, with good communications skills, are involved in community activities, are able to read and write in , are able to go to households located within two-hours walk on foot.

2.4 DATA COLLECTION/ QUALITY CONTROL Being a member of the community, the interviewers were able to collect data at the client’s convenience. The clients were not informed of the visit in advance. However, respective facility was informed of the selected patients/clients. Given the fact that health facilities do not collect complete addresses, to facilitate the tracking our verificators had to collect more information from the health care providers especially the Community Health Workers (CHWs) while selecting the patients/clients. In addition, our community interviewers contacted the CHWs for more precise address information when a patient/client could not be found.

3

For quality control, nonexistent clients were added to the list of clients selected for the survey. The interviewers were not aware of those names.

LOGIK used both electronic and paper-based data collection. For 51 health facilities including all facilities implementing RBF, the community interviewers were trained to conduct interviews and recorded data on the mobile device. The data collected was uploaded to a secure server, aggregated, and extracted in useful format. SurveyCTO which is based on the Open Data Kit (ODK) open source platform was used as a mobile data collection tool. ODK Collect uses the Android platform, supports a wide variety of prompts (text, number, location GPS), and works well without network connectivity. Therefore, it does not require mobile network access for data collection. To ensure data quality the following steps were taken:  Create skip patterns to streamline the process and avoid the possibility of skip-pattern errors.  Restrict allowed responses for individual questions to prevent entry of impossible or contradictory answers.  Use audio audits to listen in, invisibly recording audio clips at random or specific points in the survey.

For the other community interviewers who used paper-based data collection, all filled questionnaires were reviewed by the supervisors. In addition, the supervisors contacted 10% of the clients interviewed. In the departments, a supervisor was in charge of the coordination of data collection. A survey coordinator was in charge of the coordination of the data collection, quality control and carried out the analysis. Data was collected by one community interviewer for each health facility. In the metropolitan Port au Prince area, we conducted paper-based data collection due to security concerns.

2.5 DATA ANALYSIS Data analysis was carried out using Excel. A satisfaction score which is the arithmetic mean of score obtained from each respondent was calculated for each health facility assessed.

4

III. RESULTS

3.1 SURVEY POPULATION The survey was conducted in 6 geographical departments where health facilities are located: Centre, Grand’Anse, Nippes, Ouest, Sud and Sud-Est. Patients/clients were selected in all 79 health facilities included in the quantitative verification of quarter October - December 2018. The number of health facilities assessed per department, level and type is presented below in Table 1.

Table 1 : Number of health facilities assessed by department and level.

Department

Total number Centre Grand’Anse Nippes Ouest Sud Sud-Est of Health facilities Non Non Non Non Non Non Facility level RBF RBF RBF RBF RBF RBF RBF RBF RBF RBF RBF RBF CCS Dispensary 4 1 0 1 3 2 6 0 0 3 5 0 25

Health Center 1 1 0 4 1 0 24 0 0 1 1 0 33 without beds Health CS Center 3 1 0 2 3 0 3 0 0 3 2 0 17 with beds HCR HCR 0 1 0 0 1 0 2 0 0 0 0 0 4 TOTAL 8 4 0 7 8 2 35 0 0 7 8 0 79 Source: Client satisfaction Survey by VRS Central and South, April 2019

Out of 4,068 patients/clients selected, the community interviewers found 1,827. A total of 1,720 of them agreed to participate in the survey. Therefore, the response rate is 94 percent.

The tracking rate is 40% percent among clients of facilities not implementing RBF and 54% among those implementing RBF.

The majority of the respondents (78 percent) were female. Different reasons explain the female predominance. Maternal and Child Health being a priority, most of the indicators are related to that area and women use more health care services than men.

5

Figure 1: Distribution of respondents by age group

Figure 1 shows that 66 percent of the respondents were between 15 to 49 years followed by the children under age 5 (22 percent). Note that for the children who received services, the caregivers were interviewed.

Source: Client satisfaction Survey by VRS Central and South, April 2019

3.2 PATIENTS/CLIENTS TRACKING It was not possible to track 55 percent of the selected patients/clients. There were 3 most common reasons explaining that situation:  The address was incomplete (30 percent).  No one in the neighborhood including the CHWs knew the clients (24 percent).  The client lived out of the catchment area (24 percent).

The percentage of patients/clients found varied greatly from one area to another. It was easier to track clients of the departments of Sud-Est and Centre. The department of Ouest was the most difficult in terms of tracking the clients.

Figure 2: Percentage of patients tracked by department

Figure 2 shows the percentage of patients tracked for each department. It was more difficult to track patients in the Ouest department (29 percent) compared to the others.

Source: Client satisfaction Survey by VRS Central and South, April 2019

Seven percent of the clients indicated they did not receive the service recorded in the facility registers.

6

3.3 PATIENT/CLIENT SATISFACTION AT NON RBF HEALTH FACILITIES . The general level of satisfaction with care received between October - December 2018 at non RBF health facilities was high. In fact, the overall client satisfaction score was 79 out of 100 among the patients/clients of facilities not implementing RBF.

Table 2 shows that client’s satisfaction score was lower for “Waiting time”, “Opening hours” and “Payments/Fees (Formal and Informal)” compared to the other 4 items evaluated.

Table 2 : Client’s satisfaction score regarding care received between October - December 2018 at non RBF health facilities by items.

Total score Score obtained (number of clients interviewed: 1,001) Items available Centre Nippes Ouest Sud-Est All departments

1.-Quality of the reception 30 24.97 23.64 26.23 23.75 25.23 2.-Quality of care 50 40.11 47.66 45.42 37.63 42.50 3.-Availability of drugs 30 24.87 26.90 22.72 24.55 23.93 4.-Waiting time 10 5.25 7.55 6.27 7.18 6.33 5.- Opening hours 20 13.69 13.83 11.67 15.52 13.28 6.- Payments/Fees (Formal and Informal) 50 36.92 34.86 35.26 36.29 35.94 7.-Cleanliness of the health facility 10 9.95 10.00 9.77 9.91 9.85 Total 200 155.77 164.44 157.34 154.84 157.05 Satisfaction score 100 77.88 82.22 78.67 77.42 78.53 Source: Client satisfaction Survey by VRS Central and South, April 2019

The score obtained by item and the client satisfaction score for each health facility were calculated and presented in annex.

3.4 PATIENT/CLIENT SATISFACTION AT RBF HEALTH FACILITIES. The general level of satisfaction with care received between October - December 2018 at RBF health facilities was high. In fact, the overall client satisfaction score was 79 out of 100 among the patients/clients tracked.

Table 4 shows that client’s satisfaction score was lower for “Waiting time”, “Payments/Fees (Formal and Informal)” and “Opening hours” compared to the other 4 items evaluated.

7

Table 3: Client’s satisfaction score regarding care received between October - December 2018 at RBF health facilities

Total score Score obtained (number of clients interviewed: 719) Items available Centre Grand’ Anse Nippes Sud All departments 1.-Quality of the reception 30 25.35 26.11 26.04 25.86 25.84 2.-Quality of care 50 39.17 43.46 42.72 42.39 41.94 3.-Availability of drugs 30 25.46 25.27 25.79 25.26 25.44 4.-Waiting time 10 5.09 6.50 6.89 6.41 6.26 5.- Opening hours 20 16.94 15.38 14.64 15.12 15.39 6.-Payments/Fees (Formal and Informal) 50 36.42 33.89 33.00 33.26 33.79 7.-Cleanliness of the health facility 10 9.70 9.93 9.91 10.00 9.88 Total 200 158.13 160.55 158.98 158.29 158.55 Satisfaction score 100 79.06 80.28 79.49 79.14 79.28 Source: Client satisfaction Survey by VRS Central and South, April 2019

The score obtained by item and the client satisfaction score for each health facility were calculated and presented in annex.

3.5 OBSERVATION OF CLIENTS ON ACCESSIBILITY OF CARE According to the World Health Organization (WHO)2, the concept of accessibility has three dimensions: physical accessibility, financial affordability and acceptability.

Half of the respondents did not know if the health facility operates 24 hours a day. However, only 9 percent of those who had been to the health facility before their last visit found it closed.

The score for the "Payments /Fees (Formal and Informal)" score is 35 out of 50. The majority of respondents did not know the prices of the services before paying (57 percent of patients/clients) and the prices of the drugs were not displayed according to 78 percent of clients/patients.

In terms of acceptability, almost all (92 percent) of clients said they would return to the health facility if they had the same problem and 96 percent of clients would recommend the health facility to their loved ones.

2 https://www.who.int/bulletin/volumes/91/8/13-125450/en/

8

IV. MAIN ISSUES AND RECOMMENDATIONS Fifty-five percent of the patients selected could not be tracked making it difficult to issue an opinion on the first objective of the survey which is to verify that clients reported in the registers at health facilities really exist in the community and received the services. The clients could not be traced even with the support of CHWs because registers have incomplete contact details.

The difficulty to conduct spot-check on 55 percent of the patients/clients is a concern. Verification is the basis of a RBF system and the ability to trace patients/clients in the community is crucial. It is important to have registers with detailed clients contacts, addresses and phone numbers in order to facilitate tracing. In Haiti, very often an address is a series of directions and it would be difficult to record complete addresses with enough details in the registers. However, mobile phone number should be systematically collected at health facilities and for patients without phone number more details should be collected on addresses. This would facilitate patient/client tracing either by home visit or telephone.

The option of telephone interview is feasible because around two-thirds of the population have access to a mobile phone. According to the World Bank database on telecommunications3, in 2015, there were 69.9 mobile cellular subscriptions per 100 people in Haiti. Mobile cellular telephone subscriptions are subscriptions to a public mobile telephone service that provide access to the Public Switched Telephone Network (PSTN) using cellular technology. The indicator includes the number of postpaid subscriptions, and the number of active prepaid accounts (i.e. that have been used during the last three months). The indicator applies to all mobile cellular subscriptions that offer voice communications. It excludes subscriptions via data cards or USB modems, subscriptions to public mobile data services, private trunked mobile radio, telepoint, radio paging and telemetry services.

The overall perceived quality score among patients/clients found is quite high: 79 out of 100 in both health facilities not implementing RBF and implementing RBF.

The low tracking rate may introduce bias in survey findings because non-respondents may differ from respondents.

3 http://data.worldbank.org/indicator/IT.CEL.SETS.P2?locations=HT&view=chart

9