Community Survey 18 Health facilities supported by USAID through Project Santé in Centre, Ouest, Sud-Est, Nippes, Sud, and Grand-Anse departments
July 2019
VRS Central and South Project
This publication was produced at the request of the United States Agency for International Development. It was prepared independently by LOGIK.
Community Survey 18 Health facilities supported by USAID through Project Santé in Centre, Ouest, Sud-Est, Nippes, Sud, and Grand-Anse departments
DISCLAIMER This study/report is made possible by the support of the American People through the United States Agency for International Development (USAID.) The contents of this study/report are the sole responsibility of the VRS Central and South Project and do not necessarily reflect the views of USAID or the United States Government.
i
VRS Central and South Project
Contact Person: Rachelle Cassagnol, MD, MPH Chief Of Party 6, Impasse Hardy, Route du Canapé-Vert, Juvénat, Pétion-Ville, Haïti. Email: [email protected]
The USAID | HAITI VRS Central and South Project, is funded by the U.S. Agency for International Development in Haiti (USAID | HAITI) under contract no. AID-521-C-14-00009, beginning August 18, 2014. This contract is implemented by LOGIK
ii
CONTENTS Contents ...... iii List of tables ...... iv List of figures ...... iv Acronyms ...... v Executive summary ...... vi I. Introduction ...... 1 1.1 Project context ...... 1 1.2 Purpose of the community survey ...... 1 II. Survey Methodology ...... 1 2.1 Sampling ...... 1 2.2 Questionnaire ...... 3 2.3 Training of the community interviewers ...... 3 2.4 Data collection/ Quality Control ...... 3 2.5 Data analysis ...... 4 III. Results ...... 5 3.1 Survey population ...... 5 3.2 Patients/clients tracking ...... 6 3.3 Patient/client satisfaction at non RBF health FACILITIES...... 7 3.4 Patient/client satisfaction at RBF health facilities...... 7 3.5 Observation of clients on accessibility of care ...... 8 IV. Main issues and recommendations ...... 9
iii
LIST OF TABLES TABLE 1: NUMBER OF HEALTH FACILITIES ASSESSED BY DEPARTMENT AND LEVEL...... 5 TABLE 2: CLIENT’S SATISFACTION SCORE REGARDING CARE RECEIVED BETWEEN JANUARY - MARCH 2019 AT NON RBF HEALTH FACILITIES BY ITEMS...... 7 TABLE 3: CLIENT’S SATISFACTION SCORE REGARDING CARE RECEIVED BETWEEN JANUARY - MARCH 2019 AT RBF HEALTH FACILITIES ...... 8
LIST OF FIGURES FIGURE 1: DISTRIBUTION OF RESPONDENTS BY AGE GROUP ...... 6 FIGURE 2: PERCENTAGE OF PATIENTS TRACKED BY DEPARTMENT ...... 6
iv
ACRONYMS ANC: Antenatal Care
CHW: Community Health Worker
MSPP : Ministère de la Santé Publique et de la Population ODK : Open Data Kit
RBF : Results-Based Financing
USAID: U.S. Agency for International Development
VRS : Vérification des Résultats pour la Santé
WHO : World Health Organization
v
EXECUTIVE SUMMARY The overall purpose of the Verification des Résultats pour la Santé (VRS) Central and South contract is to support verification of service delivery quantity and quality outputs at sites supported by USAID through Project Santé in six geographic departments of Haiti (Centre, Ouest, Sud -Est, Nippes, Sud and Grand-Anse).
The two objectives of this contract are to: 1) Certify that facility-based performance data submitted are reliable. 2) Through periodic household surveys, certify that, at the community level, data related to health care services provided are reliable.
This patient/client survey falls under the second objective of VRS Central and South which is to certify that, at the community level, data related to health care services provided are reliable.
The survey was conducted in June and July 2019 following the quantitative verification of services provided by Project Santé-supported health facilities in the Centre, Ouest, Sud-Est, Nippes, Sud, and Grand-Anse departments during the second quarter of fiscal year 2019.
Purpose and Methodology of the survey The goals of the survey were: To verify that clients in health facility registers for the second quarter of fiscal year 2019 really exist and received services. To assess client satisfaction with respect to services rendered at the health facilities and to the management of the health facilities in general among clients who received services during the second quarter of fiscal year 2019.
VRS Central and South data verificators selected a sample of patients/clients from each health facility. Community interviewers used a questionnaire to assess client satisfaction. WE used both electronic and paper-based data collection. For 51 health facilities including all facilities implementing RBF, the community interviewers were trained to conduct interviews and recorded data on the mobile device. The data collected was uploaded to a secure server, aggregated, and extracted in useful format. SurveyCTO which is based on the Open Data Kit (ODK) open source platform was used as a mobile data collection tool.
Key Findings Out of 4,098 patients/clients selected, 1,998 (49 percent) were found. The tracking rate is 58 percent for facilities implementing RBF and 44 percent for those not implementing RBF. It was not possible to track 51 percent of the patients/clients selected due mainly to incomplete contact details in the health facility records, no one knows the clients in the neighborhood and the clients live too far from the health facilities. Among the clients tracked, the general level of satisfaction with health care received between January - March 2019 at health facilities was high as it reached a score of 80 out of 100 for both health facilities not implementing the RBF model and implementing the RBF model.
vi
For both groups, clients were less satisfied with “Waiting time”, “Payments/Fees (Formal and Informal)” and “Opening hours” compared to the other 4 items evaluated: “Quality of reception, “Perceived quality of care’’, “Availability of drugs” and “Cleanliness of health facility.”
Main issues and Recommendations Fifty-one percent of the patients selected could not be tracked making it difficult to issue an opinion on the first survey’s objective which is to verify that clients reported in the registers at health facilities really exist in the community and received the services. There is an improvement in the tracking rate especially for RBF health facilities. Nevertheless, it is difficult to conduct spot-check on 42 percent of the patients/clients at health facilities on RBF is a concern. Verification is the basis of a RBF system and the ability to trace patients/clients in the community is crucial. It is important to have registers with detailed clients contacts, addresses and phone numbers. The overall perceived quality score among patients/clients found is quite high: 80 out of 100. However, the low tracking rate may introduce bias in survey findings because non- respondents may differ from respondents.
vii
I. INTRODUCTION
1.1 PROJECT CONTEXT The overall purpose of the Verification des Résultats pour la Santé (VRS) Central and South contract is to support verification of service delivery quantity and quality outputs at sites supported by USAID through Project Santé in six geographic departments of Haiti (Centre, Ouest, Sud -Est, Nippes, Sud and Grand-Anse).
The two objectives of this contract are to: 1) Certify that facility-based performance data submitted are reliable; 2) Through periodic household surveys, certify that, at the community level, data related to health care services provided are reliable.
This report presents the results of the community survey conducted among a sample of patients/clients who received services during the quarter January - March 2019 at the above- mentioned health facilities. The survey was conducted in June and July 2019.
1.2 PURPOSE OF THE COMMUNITY SURVEY The survey falls under the second objective of VRS Central and South which is to certify that, at the community level, data related to health care services provided are reliable. The goals of the survey were: To verify that clients in health facility registers for the second quarter of fiscal year 2019 really exist and received services. To assess client satisfaction with respect to services rendered at the health facilities and to the management of the health facilities in general among clients who received services during the second quarter of fiscal year 2019.
II. SURVEY METHODOLOGY
2.1 SAMPLING During the quantitative verification covering the second quarter of fiscal year 2019 (January - March 2019), VRS Central and South verificators selected a sample of patients/clients who received selected services at the health facilities. As per UC instructions, we did not select patients/clients who received services only at the community level. A sample of 50 patients/clients was selected from each health facility among verified cases in accordance with the following sampling scheme. For dispensaries, health centers without beds and health centers with beds, the sampling was as follows: o 10 clients were selected from outpatient consultations. o 20 clients were selected from the 4 indicators with the highest bonus per unit. Depending on the services verified in each facility, the 4 indicators filling this criterion
1
may be different. Below is the list of indicators1 ranged from the highest to the smallest bonus: Number of Institutional deliveries. Number of children fully immunized children by 12 months of age. Number of bacteriologically confirmed Pulmonary Tuberculosis cases. Number of pregnant women who made at least a fourth prenatal visit during their last trimester of pregnancy. Number of cases referred to higher level health facilities. Number of pregnant women who made their first prenatal visit during their first trimester of pregnancy; Number of pregnant women tested for HIV; HIV positive pregnant women enrolled on ART; Number of HIV positive pregnant women referred to higher level health facilities; Number of TB cases treated; Number of defaulted TB cases re registered; Number of infants born to HIV positive mother placed on ARV prophylaxis within 72 hours. Number of pregnant women who received at least 2 doses of tetanus toxoid vaccine; Number of acceptors of modern Family Planning methods. Number of children between 6 and 59 months old who received Vitamin A; Number of children from 6 to 59 months old screened for acute malnutrition at the health facility; Number of children 12-59 months old dewormed.
o 20 patients were selected from the 3 indicators with the highest number of verified cases from the list of indicators mentioned above.
o An additional 10 clients were selected to replace clients that could not be tracked in the original 50 client sample. If a client in the original list of 50 could not be found, the first client in the additional list of 10 was selected to replace him/her; If a second client could not be found he/she was replaced by the second on the additional list and so on.
• For hospitals, the samples of clients were selected from the indicators verified in each facility. A sample of 100 clients/patients was selected in each hospital. If a hospital provided the services described below to less than 100 clients/patients during the quarter, all patients were selected. There are eight indicators for the hospitals: • New outpatient consultations referred by a lower level health facility. • Clients referred by a lower level facility and hospitalized. • Institutional normal deliveries. • Cesarean deliveries. • Counter referrals. • Number of TB cases treated • Number of defaulted TB cases re registered • Number of infants born to HIV positive mother placed on ARV prophylaxis within 72 hours.
1 Except the new curative consultations
2
2.2 QUESTIONNAIRE The survey questionnaire was revised by UC. UC convened a meeting on February 27, 2019 to discuss on the new verification tools and provided the survey questionnaire on February 28, 2019. The questionnaire was used during the previous verification.
The survey questionnaire includes a section on identification of the patient with his name, sex, age, address, phone number, service received, an informed consent statement and 25 questions. The informed consent statement contains the following elements: The purpose of the survey. Participation is voluntary, and refusal to participate will involve no loss of benefits to which the subject is otherwise entitled. The patient may discontinue participation without loss of benefits to which he is otherwise entitled. The extent to which confidentiality will be maintained.
The questionnaire is divided into 7 sections on client satisfaction: 1. Quality of the reception. 2. Perceived quality of care. 3. Availability of prescribed drugs at the health facility. 4. Client’s opinion on the waiting time. 5. Opening hours. 6. Opinion on Payments/Fees (Formal and Informal). 7. Cleanliness of the health facility.
The questionnaire is presented in Annex A.
2.3 TRAINING OF THE COMMUNITY INTERVIEWERS The survey coordinator trained the community interviewers to conduct data collection. The training included an overview of the VRS project, the purpose of the survey and covered interview techniques, the role of informed consent and confidentiality as well as a thorough review of the questions and the data collection tool. The training consisted of classroom presentations and mock interviews. The community interviewers are people living in each specific community being assessed, with good communications skills, are involved in community activities, are able to read and write in Haitian creole, are able to go to households located within two-hours walk on foot.
2.4 DATA COLLECTION/ QUALITY CONTROL Being a member of the community, the interviewers were able to collect data at the client’s convenience. The clients were not informed of the visit in advance. However, each facility was separately informed of the selected patients/clients. Health facilities do not collect complete addresses, making it difficult to locate the patients. Therefore, our verificators had to collect more information from the health care providers when selecting the patients/clients. In addition, our community
3 interviewers contacted the Community Health Workers (CHWs) for more precise address information when a patient/client could not be found.
For quality control, nonexistent clients were added to the list of clients selected for the survey. The interviewers were not aware of those names.
LOGIK used both electronic and paper-based data collection. For 51 health facilities including all facilities implementing RBF, the community interviewers were trained to conduct interviews and recorded data on the mobile device. The data collected was uploaded to a secure server, aggregated, and extracted in useful format. SurveyCTO which is based on the Open Data Kit (ODK) open source platform was used as a mobile data collection tool. ODK Collect uses the Android platform, supports a wide variety of prompts (text, number, location GPS), and works well without network connectivity. Therefore, it does not require mobile network access for data collection. To ensure data quality the following steps were taken: Create skip patterns to streamline the process and avoid the possibility of skip-pattern errors. Restrict allowed responses for individual questions to prevent entry of impossible or contradictory answers. Use audio audits to listen in, invisibly recording audio clips at random or specific points in the survey.
For the other community interviewers who used paper-based data collection, all filled questionnaires were reviewed by the supervisors. In addition, the supervisors contacted 10% of the clients interviewed. In the departments, a supervisor was in charge of the coordination of data collection. A survey coordinator was in charge of the coordination of the data collection, quality control and carried out the analysis. Data was collected by one community interviewer for each health facility. In the metropolitan Port-au-Prince area, we conducted paper-based data collection due to security concerns.
2.5 DATA ANALYSIS Data analysis was carried out using Excel. A satisfaction score which is the arithmetic mean of score obtained from each respondent was calculated for each health facility assessed.
4
III. RESULTS
3.1 SURVEY POPULATION The survey was conducted in 6 geographical departments where health facilities are located: Centre, Grand-Anse, Nippes, Ouest, Sud and Sud Est. Patients/clients were selected in 79 health facilities included in the quantitative verification of quarter January - March 2019. While Centre de Santé de Marmont- an RBF health facility- let us conduct the verification, we were allowed to only observe; the site did not give us access to any requested documents, registers and reports. They refused to cooperate because they do not want to be on RBF. Although we explained them that all USAID-supported sites should be verified regardless they were on RBF or not, they denied us access to the documents. We contacted the Project Santé FBR focal point for the department about the issue. Unfortunately, the problem was not solved and we did not verify any cases for Marmont and as a result there was no community survey for Marmont. The number of health facilities assessed per department, level and type is presented below in Table 1.
TABLE 1: NUMBER OF HEALTH FACILITIES ASSESSED BY DEPARTMENT AND LEVEL.
Department
Total Grand- number Centre Nippes Ouest Sud Sud-Est Anse of Health facilities Non Non Non Non Non Non Facility level RBF RBF RBF RBF RBF RBF RBF RBF RBF RBF RBF RBF CCS Dispensary 4 1 0 1 2 3 6 0 0 3 5 0 25
Health Center 1 1 0 4 0 1 24 0 0 1 1 0 33 without Health beds Center 3 1 0 2 0 3 3 0 0 3 2 0 17 CS with beds HCR HCR 0 1 0 0 0 1 2 0 0 0 0 0 4
TOTAL 8 4 0 7 2 8 35 0 0 7 8 0 79
Source: Client satisfaction Survey by VRS Central and South, July 2019
Out of 4,098 patients/clients selected, the community interviewers found 1,998. A total of 1,925 of them agreed to participate in the survey. Therefore, the response rate is 96 percent.
The tracking rate is 44% percent among clients of facilities not implementing RBF and 58% among those implementing RBF.
5
The majority of the respondents (80 percent) were female. Different reasons explain the female predominance. Maternal and Child Health being a priority, most of the indicators are related to that area and women use more health care services than men.
FIGURE 1: DISTRIBUTION OF RESPONDENTS BY AGE GROUP
Figure 1 shows that 66 percent of the respondents were between 15 to 49 years 7% Under 5 years followed by the children 22% 5 --- 14 years under age 5 (22 percent). 5% Note that for the children 15 -- 49 years who received services, the 66% > 49 years caregivers were interviewed.
Source: Client satisfaction Survey by VRS Central and South, July 2019
3.2 PATIENTS/CLIENTS TRACKING It was not possible to track 51 percent of the selected patients/clients. There were 3 most common reasons explaining that situation: The address was incomplete (36 percent). No one in the neighborhood including the CHWs knew the clients (22 percent). The client lived out of the catchment area (21 percent).
The percentage of patients/clients found varied greatly from one area to another. It was easier to track clients of the departments of Centre and Sud’Est. The department of Ouest was the most difficult in terms of tracking the clients.
FIGURE 2: PERCENTAGE OF PATIENTS TRACKED BY DEPARTMENT
Figure 2 shows the 73%
percentage of patients 67%
tracked for each 62%
53% 52% department. It was more 49%
difficult to track patients 32% in the Ouest department (32 percent) compared to the others. Centre Sud-Est Sud Nippes Grand Anse Ouest All departments
Source: Client satisfaction Survey by VRS Central and South, July 2019
Four percent of the clients indicated they did not receive the service recorded in the facility registers.
6
3.3 PATIENT/CLIENT SATISFACTION AT NON RBF HEALTH FACILITIES.
The general level of satisfaction with care received between January - March 2019 at non RBF health facilities was high. In fact, the overall client satisfaction score was 80 out of 100 among the patients/clients of facilities not implementing RBF.
Table 2 shows that client’s satisfaction score was lower for “Waiting time”, “Opening hours” and “Payments/Fees (Formal and Informal)” compared to the other 4 items evaluated.
TABLE 2: CLIENT’S SATISFACTION SCORE REGARDING CARE RECEIVED BETWEEN JANUARY - MARCH 2019 AT NON RBF HEALTH FACILITIES BY ITEMS.
Total score Score obtained (number of clients interviewed: 1,136) Items available All Centre Nippes Ouest Sud-Est departments
24.51 22.08 26.00 24.27 25.07 1.-Quality of the reception 30 37.80 48.18 45.68 40.79 43.11 2.-Quality of care 50 3.-Availability of drugs 30 26.11 26.98 22.75 25.57 24.41 6.78 7.94 6.20 7.72 6.75 4.-Waiting time 10 13.74 15.03 12.10 16.77 13.76 5.- Opening hours 20 6.- Payments/Fees 50 (Formal and Informal) 35.00 34.51 38.93 37.71 37.44 7.-Cleanliness of the 10 health facility 10.00 10.00 9.96 10.00 9.98
Total 200 153.94 164.71 161.61 162.82 160.53 Satisfaction score 100 76.97 82.36 80.81 81.41 80.26
Source: Client satisfaction Survey by VRS Central and South, July 2019
The score obtained by item and the client satisfaction score for each health facility were calculated and presented in annex.
3.4 PATIENT/CLIENT SATISFACTION AT RBF HEALTH FACILITIES. The general level of satisfaction with care received between January - March 2019 at RBF health facilities was high. In fact, the overall client satisfaction score was 80 out of 100 among the patients/clients tracked.
Table 4 shows that client’s satisfaction score was lower for “Waiting time” and “Payments/Fees (Formal and Informal)” and “Opening hours” compared to the other 4 items evaluated.
7
TABLE 3: CLIENT’S SATISFACTION SCORE REGARDING CARE RECEIVED BETWEEN JANUARY - MARCH 2019 AT RBF HEALTH FACILITIES
Total score Score obtained (number of clients interviewed: 789 ) Items available Centre Grand Nippes Sud All Anse departments 1.-Quality of the reception 30 25.72 26.23 26.14 25.56 25.91
2.-Quality of care 50 41.65 43.59 41.06 41.91 42.00 3.-Availability of drugs 25.36 25.17 26.02 25.61 25.54 30 4.-Waiting time 6.43 6.08 7.18 6.79 6.66 10 17.05 17.19 13.99 16.70 16.05 5.- Opening hours 20 6.-Payments/Fees (Formal 50 35.55 36.82 32.58 30.80 33.63 and Informal)
7.-Cleanliness of the health 10 10.00 9.89 9.81 9.95 9.91 facility
Total 200 161.75 164.97 156.77 157.32 159.70 Satisfaction score 100 80.87 82.48 78.39 78.66 79.85 Source: Client satisfaction Survey by VRS Central and South, July 2019
The score obtained by item and the client satisfaction score for each health facility were calculated and presented in annex.
3.5 OBSERVATION OF CLIENTS ON ACCESSIBILITY OF CARE According to the World Health Organization (WHO)2, the concept of accessibility has three dimensions: physical accessibility, financial affordability and acceptability.
Almost half (48 percent) of the respondents did not know if the health facility operates 24 hours a day. However, only 5 percent of those who had been to the health facility before their last visit found it closed.
The score for the "Payments /Fees (Formal and Informal)" score is 36 out of 50. More than half of respondents did not know the prices of the services before paying (52 percent of patients/clients) and the prices of the drugs were not displayed according to 77 percent of clients/patients.
In terms of acceptability, almost all (95 percent) of clients said they would return to the health facility if they had the same problem and 98 percent of clients would recommend the health facility to their loved ones.
2 https://www.who.int/bulletin/volumes/91/8/13-125450/en/
8
IV. MAIN ISSUES AND RECOMMENDATIONS Fifty-one percent of the patients selected could not be tracked making it difficult to issue an opinion on the first objective of the survey which is to verify that clients reported in the registers at health facilities really exist in the community and received the services. The clients could not be traced even with the support of CHWs because registers have incomplete contact details.
There is an improvement in the tracking rate especially for RBF health facilities. Nevertheless, we could not track 42 percent of patients/clients of health facilities on RBF. The difficulty to conduct spot-check on those patients/clients is a concern. Verification is the basis of a RBF system and the ability to trace patients/clients in the community is crucial. It is important to have registers with detailed clients contacts, addresses and phone numbers in order to facilitate tracing. In Haiti, very often an address is a series of directions and it would be difficult to record complete addresses with enough details in the registers. However, mobile phone number should be systematically collected at health facilities and for patients without phone number more details should be collected on addresses. This would facilitate patient/client tracing either by home visit or telephone.
The option of telephone interview is feasible because around two-thirds of the population have access to a mobile phone. According to the World Bank database on telecommunications3, in 2015, there were 69.9 mobile cellular subscriptions per 100 people in Haiti. Mobile cellular telephone subscriptions are subscriptions to a public mobile telephone service that provide access to the Public Switched Telephone Network (PSTN) using cellular technology. The indicator includes the number of postpaid subscriptions, and the number of active prepaid accounts (i.e. that have been used during the last three months). The indicator applies to all mobile cellular subscriptions that offer voice communications. It excludes subscriptions via data cards or USB modems, subscriptions to public mobile data services, private trunked mobile radio, telepoint, radio paging and telemetry services.
The overall perceived quality score among patients/clients found is quite high: 80 out of 100 in both health facilities not implementing RBF and implementing RBF.
The low tracking rate may introduce bias in survey findings because non-respondents may differ from respondents.
3 http://data.worldbank.org/indicator/IT.CEL.SETS.P2?locations=HT&view=chart
9