DIRTY DATA: DISEASE SURVEILLANCE IN DISTRICT, ; 3 (1) 54-67 UMU Press 2005

DIRTY DATA: DISEASE SURVEILLANCE IN , UGANDA

Driwale Alfred, Head, Maracha Health Sub-District, Arua District

Abstract

Integrated Disease Surveillance is the current modus operandi for emergency and disaster preparedness and Response in Uganda. In this article, the author presents a detailed analysis of one component of disease surveillance - reporting, as conducted in Arua District in 2001/2002. Although time has passed and changes for the better may have taken place, the article raises a number of issues and practices common to surveillance systems in other places. These include lateness of reporting, incomplete reports and most importantly, data inaccuracy. The article highlights managerial weaknesses in the management of the district health system that greatly affect disease surveillance. Finally, even before the Ugandan Ministry of Health introduced its now (in)famous 'League Tables' the author proposes models of categorising Health Sub-Districts according to performance, effectively suggesting a precursor to the current tables.

Introduction

Epidemiological Surveillance, the ongoing systematic analysis and interpretation (processing, summarisation collection, analysis, interpretation, of health data in the into relevant information for decision-making and process of describing and monitoring a health event, dissemination) and finally, feedback (communication with the objective of supporting the planning, from the data-receiving levels to the data collecting implementation and evaluation of public health points). This study mainly concentrated on the data interventions and programs (WHO, 1997), is an reporting sub system in Arua District. A system that important component of any public health system. The reports properly sends accurate and complete reports data collected usually describes the incidence of to the next level of the surveillance system in time. diseases. They may include risk factors, disabilities or health practices in the communities. Global perspectives on disease surveillance

In practice, disease surveillance is a management tool A surveillance system for diseases of epidemic essential for prediction, prevention, timely detection potential for developing countries was developed in of epidemics and response. It may also be used in India in the past decade. The system combines planning for resources, monitoring, and evaluation of Government and private sectors, with every hospital health programs and quantification of health problems. participating. The reason for the success and sustainability of this model is the simplicity of Surveillance systems commonly use data collected reporting procedure, low budget, private sector routinely in the course of patient care, laboratory participation, personal rapport with the people in the reports, surveys, and sentinel laboratory data for network, regular feedback of the information through specific diseases. Functionally, the surveillance monthly feedback, and a visible intervention upon system may be sub-divided into four main component reporting. stages. These are data collection (clinical case detection, laboratory case confirmation, and case Surveillance systems in Africa are generally weak and registration), data reporting (upward transfer of data characterized by vertical programs, delayed to the next higher level in the health system), data reporting, and low level of involvement of

health policy and development 54 volume 3 number 1 april 2005 DEVELOPMENT OF HMIS IN POOR COUNTRIES: UGANDA AS A CASE STUDY laboratories to confirm cases or outbreaks (WHO, The national health policy of Uganda provides for the 1999). In Zimbabwe, it was found that only 31% of establishment of a Health Management Information the reports from the city primary care clinics were System and a national health database (MOH, 1999). timely. The same findings were true for the rest of the This is in the hope that the system will provide an WHO member states of Africa (WHO, 1999). A study objective basis of monitoring the achievements in in Mauritius in 1996 found incomplete filling of disease prevention. This could also improve medical records generalized shortage of staff, low monitoring of epidemic and disaster preparedness, awareness among staff of the importance of the intervention against diseases targeted for eradication, system, lack of feedback and lack of utilization of control of communicable diseases, monitoring and information at the point of collection to be factors evaluation of overall sector performance. contributing towards poor compliance with the surveillance schedule, (WHO/AFRO, 1997). The first Health Sector Strategic Plan (HSSP I) translated these policy objectives into specific Due to the problems above the WHO regional office guidelines for action. As a strategy for epidemic and for Africa has proposed to strengthen the national disaster preparedness and response (E/DP&R), the Plan disease surveillance systems of the member states prescribed IDS and community-based surveillance as using an integrated approach (WHO, 1999). measures to control, manage and prevent health Implementation of the Integrated Disease Surveillance emergencies on a continuous basis (MOH, 2000). The and Response (IDSR) started in 1998 in Tanzania with main objectives of E/DP&R are detection, and prompt the rest of the countries in the region starting by 2000. response to health emergencies of public health At the time of this study, the implementation in the importance. Its target is to have a functional IDS Great Lakes Region of Africa stood at 52%. system in place by integrating disease surveillance into community health, building capacity for surveillance Disease surveillance in Uganda and maintaining a system for surveillance of endemic and epidemic emergencies (MOH, 2000). The concept of surveillance as we understand it today was introduced into the Ugandan health system through The restructuring of the national health system has the vertical programs like Expanded Programme of left it with an array of levels of administration ranging Immunization (EPI) and Control of Diarrhoeal from the national to the village level. Health units at all Diseases (CDD). They had parallel surveillance levels collect weekly and monthly surveillance data systems in addition to informal surveillance systems and prepare reports on specific pre-designed forms, for specific diseases like rabies, malaria and others. which are sent to the Health Sub-District (HSD) and Each program allocated resources specifically for later to the District office. The District office then surveillance. This inefficient use of resources and sends these reports to the MOH. For the data duplication of services also eventually increased collected to be useful, they should be analyzed and pressure on the health workers at the data collection interpreted at every level, especially at the point of points who had to respond to the requirements of each collection. They should be reported to the higher program. levels in time for appropriate and timely actions as may be required. In 1997, the Ministry of Health introduced the Health Management Information System (HMIS) with the Disease surveillance in Arua district objective of providing health managers with information for decision-making. This enabled the Arua district, located in the north-western West information about diseases of national interest to be region of Uganda, had seven HSDs (Ayivu, , Arua sent in one report. In 1998, member states of the Municipality, Maracha, , Terego, and Lower African region of the World Health Organization Madi) and 62 health units at the time of the study. In (WHO-AFRO), Uganda inclusive, adopted the order to strengthen disease surveillance, the district Integrated Disease Surveillance (IDS) strategy to had undertaken specific capacity building measures at strengthen the national infectious disease surveillance the different levels of the health system like training system with the main objective of early detection and HMIS focal persons for each HSD and some health response to communicable diseases. This strategy unit staff, and recruiting Records Assistants for the aimed at optimizing the use of scarce resources health units. Specific budgetary provisions were also targeted for surveillance. This eventually gave rise to made to ensure the availability of some resources for the current Integrated Disease Surveillance (IDS) surveillance activities. system in all .

health policy and development 55 volume 3 number 1 april 2005 Driwale Alfred

Problem statement Methods

Although there were significant improvements in the At health unit, Health Sub-District and District levels, reporting of HMIS data nationally, with the national we reviewed the records of two HMIS reports average scores of the weekly surveillance being 82% (monthly report and weekly surveillance report) for for timeliness and 86% for completeness, Arua the year 2001/2002 and interviewed the health work- district was performing far below the national target ers responsible for the HMIS to identify the factors of 80% (MOH, 2002). Between March 2001 and influencing data reporting. Assuming 60% of the units March 2002, only 39% of the Weekly Surveillance to be reporting to the District level and accepting a Reports from of the health units in Arua district had margin of error of 5% at the 95% level of confidence, reached the MOH. Although the reporting structures we determined that the acceptable minimum number for surveillance existed, the District Health of records representing the entire study population of Management Team (DHMT) had expressed records would be 384 records of each report by using dissatisfaction with the reporting of HMIS data the formula [sample size = P (100-p)/d2]. Since we during their supervision meetings. Money had been had seven Health Sub-Districts, we got a minimum of spent on the training of health unit staff and district 55 monthly reports and 55 weekly surveillance reports focal persons and data was being collected in health per Health Sub-District for the whole year, or a units. The performance scores above seemed to minimum of 5 records per HSD per month. We suggest some operational difficulties. There was the analysed 6 records of each form per HSD per month. risk of an epidemic occurring unnoticed and taking the district health authorities too long to respond. It The study variables were timeliness, completeness, was thought that information obtained with this study accuracy and the factors that affect data reporting as could be useful for better management of the district detailed below. We assessed performance in each of disease surveillance system, especially because there these variables against the standards set by the were already other initiatives and programs aimed at Ministry of Health. Regarding timeliness, the ministry eradication of diseases such as polio, guinea worms, required that 80% of the expected weekly surveillance onchocerciasis, etc. reports should have reached the district by Tuesday of every subsequent week and that 80% of the The aim and objectives of the study expected monthly reports should have reached the district level by the 14th day of every subsequent The study aimed at assessing the reporting month. For completeness, the ministry expected two component of the disease surveillance system in the approaches. First, there was completeness in the sense district (HMIS and weekly surveillance) in the of having enough reports and, second, there was financial 2001/2002. Assessing the timeliness, completeness in the sense of having the report forms completeness, accuracy and the reasons for the poor fully filled in. The first parameter measured complete- performance in these parameters could help the ness at the district level (80% of expected reports district tackle their root causes and help improve their received from health units by dates above) while the preparedness. The objectives were second parameter measured completeness at health unit level (80% of the forms completely filled in at the 1. To determine the completeness at health unit and health units). We assessed accuracy by comparing the district levels of the monthly health unit reports individual reports with the data sources (case and Weekly Surveillance reports against the registers) at the health units for concordance. For the national standards for HMIS completeness accuracy of the monthly reports, we used records of two common and killer diseases (malaria and diarrhoea) 2. To determine the timeliness of weekly and and the records of two common notifiable diseases monthly reporting to HSD and District levels (dysentery and measles) to verify the accuracy of against the national standards for HMIS timeliness Weekly Surveillance Reports. We randomly selected two months (October 2001 and April 2002) and three 3. To determine the accuracy of the weekly HUs per HSD for this exercise. surveillance and monthly reports and We obtained the factors influencing data reporting by 4. To determine the factors influencing the reporting interviewing key people involved in the management of data at the levels of the district health system of data at all levels in the district using an interview guide and checklists. We interviewed all the members of the District Health Team and the individual Health

health policy and development 56 volume 3 number 1 april 2005 DIRTY DATA: DISEASE SURVEILLANCE IN ARUA DISTRICT, UGANDA

Sub-District Management Teams. We modified Table Against a national target of 80% Completeness per 2 of the WHO checklist for surveillance and used the month at the district office, the performance by the modified version to guide the interview. While at the HSDs is shown in Graph 1 below. HSD, we tracked the late reports earlier identified at the district office in order to verify the points of delay in the health system. We tracked one late report per randomly sampled health unit and interviewed the person in charge of the unit in charge, the senior nursing officer and the records assistant for an explanation for the delay. We studied late reports from 21 lower level health units (3 per HSD) and all hospitals and HC IV.

Findings

Completeness at district level

The number of monthly reports received was expressed as a percentage of the reports expected from the health facilities in that month. However, during data collection, the reports for the month of June had not been received at the district. The district an overall mean completeness of 83% (median 100%), just above the national target of 80% and that among the Health Most of the sub-districts met the national target for Sub-Districts, Maracha HSD had the best mean completeness at 80%. completeness at 97% (median 100%). The worst

Table 1: Percentage completeness of monthly reports per HSD (July 2001 to May 2002) Health Sub-District Monthly Performance in Completeness (% of expected reports received) AMC AYIVU KOBOKO L/MADI MARACHA TEREGO VURRA Mean Median Number 8 6 5 9 8 12 14 of units JUL 29 67 20 0 100 77 79 53 67 AUG 57 100 80 89 88 69 100 83 88 SEP 59 100 100 89 100 62 100 87 100 OCT 86 100 100 89 100 77 93 92 93 NOV 71 100 60 89 100 54 100 82 89 DEC 43 100 80 100 100 54 79 79 80 JAN 86 83 100 100 75 54 64 80 83 FEB 86 83 80 100 100 85 100 91 86 MAR 57 100 100 89 100 77 79 86 89 APR 71 100 100 89 100 69 100 90 100 MAY 86 83 100 100 100 69 100 91 100 Mean 66 92 84 85 97 68 90 83 Median 71 100 100 89 100 69 100 100 performer was Arua Municipality. The District had had For completeness of the Weekly Surveillance Forms its best performance in completeness during the months submitted to the DDHS' office, we counted the of September 2001, April 2002 and May 2002 reports received per month and expressed them as a (Median 100%) and the worst performance in July percentage of those expected in that month. This is 2001. summarised in table 2.

health policy and development 57 volume 3 number 1 april 2005 Driwale Alfred

Table 2: Completeness of weekly surveillance reports per HSD % completeness per HSD MONTH AMC AYIVU KOBOKO L/MADI MARACHA TEREGO VURRA Mean JUL 22 25 40 8 0 35 9 19 AUG 0 8 15 3 0 19 25 12 SEP 16 46 15 3 0 27 27 19 OCT 3 63 65 14 63 33 16 32 NOV 3 4 5 11 0 6 23 9 DEC 19 8 10 17 6 46 2 17 JAN 0 33 35 25 31 25 0 19 FEB 6 0 0 0 0 2 0 1 MAR 116 83 140 72 91 46 32 73 APR 56 46 25 39 41 38 48 43 MAY 78 96 80 67 106 81 73 81 JUNE 128 54 85 81 106 27 79 77 Mean 27 34 36 22 28 30 21 27 Median 18 40 30 36 19 30 24 24

Over the study period, the overall mean monthly (range: 21% - 36%) and a median of 24% (range: 18% completeness of the Weekly Surveillance Forms was - 40%) 27% ranging from 1% in the month of February to 81% in May 2002. There was a general tendency to Completeness at health unit level improve in reporting to wards the end of the financial year as shown by the moving average in Graph 2 We assessed the completeness of the monthly reports below. In general, the compliance of the individual HSD using the second aspect of completeness, the filling in in submitting the Weekly Surveillance Forms was not of all the provided spaces at health unit level. only very poor but also fluctuant, with a mean of 27%

Table 3: Completeness of weekly surveillance reports per HSD % completeness per HSD MONTH AMC AYIVU KOBOKO L/MADI MARACHA TEREGO VURRA Mean JUL 0 33 16 66 83 16 50 38 AUG 33 100 33 100 66 66 83 69 SEP 50 100 50 66 100 16 100 69 OCT 50 100 16 100 66 33 100 66 NOV 50 83 16 83 66 0 33 47 DEC 50 66 33 83 100 0 66 57 JAN 66 50 66 50 50 83 66 62 FEB 83 66 33 66 66 100 83 71 MAR 66 100 50 66 83 33 50 64 APR 83 66 50 83 83 83 83 76 MAY 66 33 50 83 83 50 50 59 MEAN 54 72 38 77 77 44 69 62

On average only 62% of the reports sent by health 76% throughout the year. Graph 2 illustrates this facilities were complete, varying between 38% and monthly variation.

health policy and development 58 volume 3 number 1 april 2005 DIRTY DATA: DISEASE SURVEILLANCE IN ARUA DISTRICT, UGANDA

Graph 2: Completeness of the monthly report at health facility by month

The level of completeness varied greatly among the ranging from 38% to 77% as shown in Graph 3 HSDs. A wide difference existed between them, below.

There were more serious variations in the of the weekly surveillance system in the month of Feb- completeness of the weekly report over time as shown ruary 2002. in Graph 4 below. There was even a total breakdown

Graph 4: Completeness of weekly surveillance per month.

health policy and development 59 volume 3 number 1 april 2005 Driwale Alfred

Timeliness at district level

We assessed the timeliness of reporting to the district reports received at the District office by the required by determining the percentage of of the monthly date as shown in Table 4 below.

Table 4: Timeliness monthly reports at the district office Percentage of timely reports per HSD per month MONTH AMC AYIVU KOBOKO L/MADI MARACHA TEREGO VURRA Mean JUL 33 66 0 0 66 33 66 38 AUG 16 16 50 100 100 100 16 57 SEP 0 33 16 33 100 83 33 43 OCT 33 66 83 66 100 100 83 76 NOV 0 100 0 0 100 50 33 40 DEC 33 33 66 0 100 100 66 57 JAN 33 66 0 0 0 83 100 40 FEB 83 0 33 50 100 83 66 59 MAR 0 33 0 100 100 83 100 59 APR 33 0 83 100 100 100 100 74 MAY 0 0 83 100 100 66 66 59 MEAN 24 38 38 50 88 80 66 55

On average, over the study period, 55% (range: 38 - report at the district to the date on which the reports 76%) of the monthly reports arrived at the district in were expected. This is the most important report as time as expected. There was no particular monthly far as the surveillance for new disease outbreaks is trend in timeliness. However, the timeliness per sub- concerned and it must be reported accurately and district varied greatly with average timeliness ranging quickly. The data for the weekly report are colleted from 24 to 88%. Arua Municipality HSD had the worst every Monday and the report must be at the district timeliness average despite being at the District head- on Tuesday, ready for transmission to the Ministry quarters. every Wednesday. The table below shows the percentage of the reports which were received on time We also assessed the timeliness of weekly surveillance per HSD during the period under study. reporting by comparing the date of receipt of the Table 5: Timeliness of weekly surveillance reports per HSD at the district % of reports timely per HSD MONTH AMC AYIVU KOBOKO L/MADI MARACHA TEREGO VURRA Mean H/Us 8 6 5 9 8 12 14 JUL 71 33 63 100 0 53 100 60 AUG 0 50 67 100 0 100 100 60 SEP 100 36 67 100 0 85 100 70 OCT 100 33 46 20 70 50 100 60 NOV 100 100 100 100 0 100 100 86 DEC 100 0 0 50 50 64 100 52 JAN 0 75 100 100 100 100 0 68 FEB 0 0 0 0 0 0 0 0 MAR 86 60 61 100 90 59 100 79 APR 100 100 100 100 38 61 100 86 MAY 72 96 56 100 71 46 100 77 JUNE 78 77 71 100 79 46 100 79 Mean 67 55 61 81 42 64 83 65 Median 82 55 65 100 44 60 100 69

health policy and development 60 volume 3 number 1 april 2005 DIRTY DATA: DISEASE SURVEILLANCE IN ARUA DISTRICT, UGANDA

The timeliness of this report was generally poor. . The median is a more reliable indicator because there Compared to the national target of 100%, the were extreme values (0 to 100) observed. During the timeliness per HSD was only of a median of 69% month of February 2002, there was a complete break (range: 44 - 100%) and an average of 65% (range: 42 down and no report was received on time from all the - 83%), with Maracha HSD being the slowest reporter HSDs. These trends are shown in Graph 3.10 below.

Graph 3.10 Timeliness for weekly surveillance per month

Accuracy of reporting

The accuracy of a report tells the manager whether selected HUs (3 per HSD) by cross-checking the the report should be acted upon or not. It makes the reported records against the registered records of two manager decide whether to deploy resources to indicator diagnoses responsible for the highest contain a situation or not and whether any action morbidity rates. There were general inaccuracies in should be taken or not. The accuracy of reporting the reporting. The findings per HSD were aggregated was assessed in the OPD records of 21 randomly and are presented in Table 6 below.

Table 6: Accuracy of the monthly report for two diagnoses

MALARIA PNEUMONIA HSD Reported Registered Inaccuracy Reported Registered Inaccuracy AMC 4,377 3811 +15% 785 689 +14% AYIVU 3,718 3540 +5% 192 163 +18% KOBOKO 758 716 +6% 113 92 +23% L/MADI 2,754 2593 +%6 46 33 +39% MARACHA 2,458 2335 +5% 291 269 +8% TEREGO 1,356 1260 +8% 215 229 -6% VURRA 982 1163 -16% 347 337 +3% DISTRICT 16,403 15,418 +6% 1,989 1,812 +10% + sign = over-reporting - sign = under-reporting

health policy and development 61 volume 3 number 1 april 2005 Driwale Alfred

There was a general tendency for over-reporting cases from the clinical care of patients. This worsens the in all HSD for the selected indicator diagnoses. For already constrained staffing situation in the health units. both, there was an average inaccuracy of +8% (or the The clinical staff not involved in surveillance activities reports were only 92% accurate). In general, Lower expressed their frustration at the amount of work they Madi HSD tended to over-report while Terego HSD have to do alone, since selection of focal persons has tended to under-report. For weekly reporting, the been classified as 'technical work' requiring somebody numbers of reported cases in the reporting forms were with a technical background in health issues. to be matched with the numbers in the case register at the facility. However, in most units we could not trace Managerial factors: In most of the units visited, the source registers and the validity of the reports were forms were only filled by either the person in charge highly doubtful. Attempts to triangulate the reports with of the health unit or the Records Assistant. The the OPD registers were not successful because the duty was never delegated to any other person and numbers of cases were so few that the percentage whenever they were out of station, the reports were errors appeared exaggerated. Hence, collection of never compiled. This could have a bearing to the per- this data proved difficult and was subsequently ception of heavy work involved or to the expectation abandoned. of allowances for the task. This ultimately affected the timeliness and completeness of reporting. In some cases, there was clear evidence of procrastination. We What factors influence the reporting system? found that in 75% of the units studied, records were not analysed on daily basis as recommended. Tallying As shown above, data reporting in the district was was only done at the end of a month at the time of below the national target of 80% for both timeliness preparation of the report. This was done hurriedly and and completeness and in some months there was probably contributed to the tallying errors because of a complete break down of the system. We sought the heavy workload to be completed in a short time. for the factors that constrained the process of data reporting by interviewing the health workers. These Staff accommodation: Two of the health units (10%) were mainly the shortage of personnel leading to an studied had no accommodation for staff at all. The overall heavy workload, poor managerial skills, lack staff were staying about 10 Km away from the health of accommodation, lack of transport, lack of an facilities and a substantial amount of their time was innovative spirit, lack of computer training and lack spent on travelling to the health units and back home. of funds. This affected the output at the workplace in clinical work and in data management. Workload: When interviewed about the ability to fill the forms, the staff reported finding no difficulties Means of transport: Of the health units studied, only with filling any of the two forms under study. They 35% had means of transport in form of a car or affirmed that the forms were easy to understand and motorcycle. However, all the HSDs have either a car simple enough to fill. However, the health units are or a motorcycle that could be used for surveillance. under-staffed and have difficulties in coping with the The main problem with transport was therefore the workload of the health facilities. Due to the shortage fear of excessive running costs for the HSD if the of staff, a single health worker often has to register vehicles were regularly used for reporting and data patients, take history and examine them, dispense collection. In a few units, the means of transport drugs, give injections and dress wounds etc, all by were used for multiple purposes, competing more himself or herself. With no Records Assistants in some favourably against disease surveillance. In any case, of the HUs and no specific incentives to do the duty, most of the vehicles were old and had frequent break- surveillance is often looked at as an extra burden. downs and frequently led to simultaneous breakdown of the reporting system e.g. Koboko, Terego and L/ Besides the time spent on compiling the reports, most Madi HSDs. The HSDs also cited irregular and unreli- HSDs spend one day collecting the weekly able supply of fuel by the district headquarters as a surveillance reports from the health units and one other common constraint. day to take them to the office of the DDHS; hence two person-days per HSD per week. Given that the Communication: Transmission of the surveillance position of 'HMIS focal person' is not established, it reports from one level to another was by hand means that this is a duty combined with other official delivery. No health HSD used any other means even duties of the person. This and other issues when the option for doing so existed to varying individually or in combination take away critical staff extents e.g. telephone, radio, mail. All the HSDs

health policy and development 62 volume 3 number 1 april 2005 DIRTY DATA: DISEASE SURVEILLANCE IN ARUA DISTRICT, UGANDA had networks for mobile cellular telephones in their highest among the sampled units. Some irregularities catchment area and 5/7 (71.4% ) had radio call were observed in the selection of the health units and facilities. None of the HSDs had ever used either the the participants for the training. Two of the sampled telephone or the radio call to communicate units (10%) were not involved in the training at all, surveillance data to the district. Use of these facilities e.g. Ludara and Oriajini. This could not be explained would be cheaper, faster and markedly improve the by a limit in the number of participants the district performance of data reporting in the district. This was capable of training because some neighbouring option could be used with the aim of improving time- health units (Koboko HC IV and Wandi HC III) had liness but not as a replacement alternative to the use sent three participants each. In fact, staff reported of reporting forms, which could still be sent for record lack of skills in analysis of data and drawing of graphs purposes. Apart from the communication media, some as their major weakness. This could have suggested a HSDs lacked a clear policy on the flow of reports weakness in the curriculum of the training but it was covering compilation of the data for the reports and not possible to get a copy of it. Although all the HSD collection of the reports after they were filled. In some offices in the district had computers, none was used HSDs, some units would take their reports directly to for data management, not even at the district office. the district office, bypassing the HSD and there Lack of computer skills was reported to be the main was no agreed mechanism for the HSD to get its reason for not using them. copy. Staff attitudes: Many staff saw disease surveillance Financial resources: Given the procedures for data as a vertical program and extra work that should be reporting in the district there was little need for the done for an allowance. This was mainly observed individual health units to budget for reporting, as their in Maracha and Vurra HSDs but could have been role is limited to data compilation within the facility. prevalent in others areas as well. In fact, all the HSDs The stationery was provided by the district and was reported cases where at some health units, staff were never out of stock in all the units for all the study reluctant to fill the forms and the focal persons had to period. The data were collected by the HSD. There- fill them out themselves when they arrived for data fore, there should have been few financial constraints collection. This delayed the focal persons and for that level. In fact, the staff at the lower level health prolonged the data collection process. It had a units raised no complaint about finances being a significant effect on the timeliness of reporting to the barrier to data reporting. The budget for the activity district. For some, reports of 'zero' cases were given should have been at the HSD level. However, to 'make life easy'. Such an attitude would have a none of the HSDs had a budget specific for disease profound effect on accuracy of reports and detection surveillance activities. However, they had budgets for of epidemics. Finally, due to the geographical disaster preparedness and response which where in distribution of the units, some units reported directly practice reallocated for surveillance. For the most part, to the district office, by-passing the HSD. This meant these were small amounts of money, which could not that, unlike the HMIS focal person for the entire HSD sustain the surveillance activities throughout the who was paid an allowance, the health unit staff did financial year. The highest amount was Uganda not receive any payment. Such irregularities in Shillings 890,000/= (about US $ 520) budgeted by payments were found to be silently eroding the Lower Madi HSD. The budgets of the other HSDs morale of the staff. On their part, Records Assistants were much lower. In Lower Madi itself, the shortest complained of unclear job descriptions and lack of a distance possible to be covered by a focal person clear career path. collecting reports and delivering them to the district office was 370km. Demarcation of the HSDs: Finally, owing to the large size of some of the HSDs and the difficulties faced in Staff training: About 80% of the people involved managing them from the HC IV, some HSDs were responsible for filling the forms in the health facilities re-demarcated. The HSDs of Vurra, Madi, and Terego studied had received some training in the exercise. did not correspond to their political constituencies. Exceptions were in Arua Regional Referral Hospital Partly due to this, we found that some of the HSDs and Oriajini Hospital. However, some of the people did not to have a good mastery of the HUs under their who had been trained were not involved in compiling care. For instance, Siripi HC originally under Terego the surveillance forms. The consequences of not but now under Lower Madi had accumulated weekly training the Records Assistant for a big hospital such surveillance reports for three months. Neither HSD as Arua could probably explain the high level of had collected them. This raised questions of who inaccuracy (+15%) seen in the hospital data - the supervised the unit in other matters.

health policy and development 63 volume 3 number 1 april 2005 Driwale Alfred

Data Management at higher levels of the District refers to the number of health units which have Health System reported for that month. At health unit level however, 'completeness' refers to the parts of the report that Despite the presence of HMIS focal persons at the are filled. Second, the range of activities required for HSDs, there was no attempt at that level to combine completeness at health unit level may not all be the individual health unit reports to get the overall conducted at a particular health unit and, third, the picture. At district level, this was done, though to a workload of completing the report at health unit level limited extent. The manner of filing the reports at the may be too much. The monthly report (HMIS 105) district made their retrieval difficult, with many has eight parts to complete: OPD and Laboratory tests, reports getting lost. This was beginning to raise OPD diagnosis, ANC and FP activities, essential drugs, complaints especially from staff in Terego and Vurra vaccines and contraceptives, outreach activities, HSDs. Handling of reports at the higher offices financial summary, comments, and monthly affects the quality of the eventual feedback given to monitoring. Some units do not provide some expected the HSDs and the health units. services. Activities that were not carried out by a unit either due to its level or due to lack of equipment or Points of delay staff should have been indicated on the reporting forms. Failure to do so meant incomplete reporting. Of the reports, which eventually reached the district In fact, the section on financial reports was excluded office, 29% were late despite favourable dates of in the assessment since the majority of the units either compilation at health unit level. Exploration suggested did not handle cash or left it blank. However, one that significant delays were occasioned by the health cannot escape the fact that even when services were sub-district level, which either failed to collect the provided, the appropriate sections were not filled in reports from the health units or failed to submit the some units. reports to the district office at the right time. In general, we observed a lack of capacity to handle data There was no uniform understanding of how to fill collection and reporting at the HSD level. In addition, the form and this could leave the filling open to some HSDs were too large to be covered in one day. multiple interpretation. For instance, putting a dash Most have between 8 and 14 health units and, that sign (-), a zero, or just leaving the section blank could apart, the units were widely scattered geographically all mean the same thing or carry significantly different and did not lie along a single common route for easy meanings to different people in different units. In most collection of the forms. units, the dash on a form could mean that the services indicated on the form were not offered at the facility. Zeros could mean that the services indicated were DISCUSSION available but had not been consumed in the reporting period. A blank space was the one most open to Completeness of the reports multiple interpretations since it could mean any of the two former signs. In reality, all the three responses In general, the completeness of monthly reporting could mean any one of the three situations. Such a was better than that for weekly reporting and the detail, trivial as it may sound, grossly affected the completeness of monthly reporting was better than interpretation and therefore the quality of the for weekly reporting. However, we observed some feedback to the health units from the higher levels of occasional irregularities in this positive trend. Although surveillance. Theoretically, incomplete reports from the completeness of reporting at HSD was generally health facilities can be corrected at the level of the acceptable, the performance of two HSDs of Terego HSD by the focal person after thorough verification and Arua Municipal Council, which missed the HSSP with the source unit. In practice, however, these target for most of the year, deserved investigation. people were also the data recording persons for the The management of data even at district level was not HSD units and given a tight due date schedule, did not as good as should be expected, for, data from one do thorough scrutiny of the reports from units other HSD were missing despite evidence of the reports than theirs. having been received. For the weekly surveillance report, a monthly average The completeness of the reports at facility level was completeness of 27% for the district was far below much lower than that at district level because of a the national HSSP target of 80%. Even the best HSD number of factors. First, they measure different in completeness (Koboko) did not reach this target. aspects of completeness. Completeness at district level However, a dramatic improvement was seen in the

health policy and development 64 volume 3 number 1 april 2005 DIRTY DATA: DISEASE SURVEILLANCE IN ARUA DISTRICT, UGANDA last four months of the financial year, thanks to the However, if they are 'dirty' by virtue of inaccuracies, training of focal persons and record assistants and the then they are useless, misleading and could be dan- graphic feedback given to the HSDs by the District gerous. Using only two common diagnoses, we found office. that most data were inaccurate by a significant margin. One therefore wonders what the correlation We learnt two practical and important lessons from would be if we were to verify the records of less the improvement seen above, namely, that training common diagnoses or to go further and verify the improves performance and, more importantly, that correctness of the diagnoses. In Arua District, the feedback given influences reporting by setting a spirit management of records was quite poor. Retrieval of of competition and motivation. In fact, the units wanted the old registers for the selected months was a big unit specific feedback in a graphic manner where they constraint in some HSDs. Most registers had already can compare their performance against that of other been dismantled and used for other purposes and as a units and against themselves over time. consequence, in one HSD the verification exercise was limited to one health unit out of the expected three. Before March 2002, no HSD met the completeness Some health facilities used more than one register at a criterion in the district and, in fact, there was a total time. This posed a difficulty in retrieving them. breakdown of the surveillance system with 1% of the Despite the difficulties faced the results were very units reporting in the month of February. Surveillance revealing; agreeing very closely with the averages of reports for the first quarter of the financial year were the other HSDs where the required records were not received for Maracha HSD. This was partly blamed assessed. In general, the accuracy of reporting by the on the loss of reports resulting from the filing defi- HSDs and the units was low with an average error of ciencies at the district level. 7%. Over-reporting was the commonest type of error found in the health units which were sampled except Timeliness of reports in Vurra where there was under-reporting.

The principle behind timeliness of reporting is timely There were no reliable sources to even test the detection and response of the health authorities to the accuracy of the weekly surveillance forms. This identified health event. The indicator of good shows the great danger in which the district was at performance for timeliness is that at least 80% of the the time of the study, of being caught unawares by reports are received timely at every level of the health an epidemic. If there were no reliable data sources system (HSSP 1999). In Arua, only 55% of the for this report, we doubt if the report itself was reports were received in time for timely transmission reliable. to the Ministry. Most HSDs hardly reached the 40% mark in timeliness and yet the timeliness of reporting A composite comparison parameter to the district is the determinant of the districts' completeness at the national level. The findings above A good surveillance system should have complete suggest that a lot of effort was still required to rectify reports, which are received in time at every level, and the timeliness of reporting in the district in order to the reports should be accurate. From the preceding, achieve the said purpose. we saw that HSDs differ in performance on all the parameters. Furthermore, within a HSD the On average, the district received only 65% of the performance with respect to the parameters assessed weekly surveillance reports in time, which is below varies greatly from health unit to health unit. the HSSP target for timeliness of 80%. It raises false However, in order to compare the different HSDs (and hope to imagine that 65% is near the target. However, later the different health units), a composite measure the critical point is that the district receives only 27% for all the aspects of reporting above needs to be of the expected reports from the health units, and only obtained and used to rank them into positions- a form 65% of these (27%) reports arrive in time. This means of league table. We attempted this by multiplying the that, on average, only data from a mere 18% of the average scores obtained by each HSD in each health units could be immediately useful for planning parameter and dividing the product of these scores by response to epidemics in time. the product of the maximum score obtainable i.e. 100%. The final quotient is then expressed as a percentage Accuracy of Reporting (the efficiency). We do not claim that the effect is necessarily multiplicative, for it could be additive or If health data are late or incomplete but accurate, a otherwise but this was the only reasonable health manager can still use them for other purposes. mechanism we thought could capture synergistic

health policy and development 65 volume 3 number 1 april 2005 Driwale Alfred parameters of the same effect - the efficiency. We do surveillance, the accuracy data of the monthly HMIS not even claim that the three parameters have the same 105 form were used for both reports. After all the weight. degrees of error demonstrated were minimal and would not have changed the over all scores signifi- The values obtained can be a measure of efficiency of cantly. This could be modified in districts with good data reporting. Owing to the irregularities in data sources. the collection of data for accuracy for the weekly

Table 7: Estimated HSD Efficiency in monthly reporting (HMIS 105) HSD Average Average Average Average POSITION/ Completeness Timeliness Accuracy Efficiency RANK (%) (%) (%) (%) Maracha 97 88 95 81 1 Terego 68 80 92 50 2 Vurra 90 66 84 49 3 L/Madi 85 50 95 40 4 Ayivu 92 38 95 33 5 Koboko 84 38 94 30 6 AMC 66 24 85 14 7

Using the same reasoning as above, we worked out the ranks for weekly reporting.

Table 8: HSD Efficiency in Weekly reporting HSD Average Average Average Average POSITION/ Completeness Timeliness Accuracy Efficiency RANK (%) (%) (%) (%) Koboko 36 61 93 20.4 1 Terego 30 64 93 17.9 2 AMC 27 70 93 17.6 3 L/Madi 22 81 93 16.6 4 Vurra 21 83 93 16.2 5 Ayivu 34 50 93 15.8 6 Maracha 28 42 93 10.9 7

'Completeness' in both cases refers to completeness at the district office.

These tables highlight the strengths of the HSDs with surveillance forms: the monthly (HMIS 105) respect to data reporting. We think that this could be a and weekly surveillance forms. We assessed some useful tool to be used for effective feedback to parameters of performance such as, completeness, the HSDs and which could serve as a guide to timeliness and accuracy as well as the factors the revitalisation of effective data reporting. affecting them. We observed that whereas the completeness of the monthly form at the district was Conclusions acceptable, that of the weekly surveillance form was unacceptably low and with occasional breakdowns. This study set out to establish the performance of the The majority of the reports were received at the integrated disease surveillance system in Arua District, district in time but were inaccurate, with a tendency with respect to the reporting of the two commonest for over-reporting figures. However, there were wide

health policy and development 66 volume 3 number 1 april 2005 DIRTY DATA: DISEASE SURVEILLANCE IN ARUA DISTRICT, UGANDA differences between the HSDs of the district with References respect to completeness, timeliness and accuracy. Multiple factors affected the performance of data District Director of Health Services, Arua, 2001; 2001/ reporting in the district, the majority being financial 2002 Arua District PHC Work Plan. and administrative. However, even with the resources available then, there was still room for improvement. Ministry of Health (Uganda), 1999; The National Health As a general conclusion, the integrated disease Policy surveillance system of the district was ineffective and Ministry of Health (Uganda), 2000; The Health Sector placed the population in danger of an undetected Strategic Plan (1999 - 2004) disease outbreak if left unaddressed. Ministry of Health (Uganda), 2001; Highlights Of Sector The key issues to be addressed were human resources, Performance For Financial Year 2001, Brochure planning, and management. There was a need for recruitment or designation and training of staff as Ministry of Health (Uganda), 2001; Indicators For Records Assistants in all the health units. The Monitoring Health Indices And The Health Sector Stra- available means of communication needed to be fully tegic Plan In Uganda exploited and utilised for timely transmission of reports. There was a need for rendering somebody Ministry of Health, (Uganda) 2000; Planning Guide- responsible for data aggregation, analysis and lines For Disease Surveillance, Epidemic Preparedness feedback at district level. In order to improve on data And Response For The District storage and retrieval at all the levels of the district, WHO, 2000; The Current Status Of IDSR In The there was a need for training HSD HMIS focal staff in Great Lacks Region; IDS/ Health Information Bulletin computer literacy and skills to be able to enter, retrieve and analyse HMIS data. WHO, 2001, W.H.O Cooperation In Strenghthening National Health Information Systems

WHO, 2002; Improving EPI Performance In Uganda, IDS/ Health Information Bulletin

WHO/EMC/DIS/97.2, 1997, The Protocol For Evalu- ation Of Epidemiological Surveillance Systems

health policy and development 67 volume 3 number 1 april 2005