UNITED NATIONS HIGH COMMISSIONER FOR REFUGEES EVALUATION AND POLICY ANALYSIS UNIT

The road to health and the road to Afghanistan

Joint UNHCR/WHO evaluation of health and health programmes for Afghan refugees in Pakistan

Markus Michael, Mary Corbett, and EPAU/2005/05 Glen Mola April 2005 Evaluation and Policy Analysis Unit

UNHCR’s Evaluation and Policy Analysis Unit (EPAU) is committed to the systematic examination and assessment of UNHCR projects, programmes, policies and practices. EPAU also promotes rigorous research on issues related to the work of UNHCR and encourages an active exchange of ideas and information between humanitarian practitioners, policymakers and the academic community. All of these activities are undertaken with the purpose of enhancing UNHCR’s capacity to fulfill its mandate on behalf of refugees and other people of concern to the organization. The work of the unit is guided by the principles of transparency, independence, consultation and relevance.

Evaluation and Policy Analysis Unit United Nations High Commissioner for Refugees Case postale 2500 CH-1211 Geneva 2 Depot Switzerland

Tel: (41 22) 739 8249 Fax: (41 22) 739 7344

e-mail: [email protected]

www.unhcr.org

All EPAU evaluation reports are placed in the public domain. Electronic versions are posted on the UNHCR website and hard copies can be obtained by contacting EPAU. They may be quoted, cited and copied, provided that the source is acknowledged. The views expressed in EPAU publications are not necessarily those of UNHCR. The designations and maps used do not imply the expression of any opinion or recognition on the part of UNHCR concerning the legal status of a territory or of its authorities.

This evaluation report is of one of a series of three health evaluations conducted in late 2003 among refugee populations. The purpose of these evaluations is not only to improve health service delivery to those populations, but also to act as pilot evaluations for a broader "Inter-agency Health Services Evaluation in Humanitarian Crises" initiative. This initiative was launched by a group of non-governmental organizations, UN agencies, and academic institutions, and was stimulated by the lack of coherent, routine evaluations of the health of affected populations in crises. It has received funds from the US Bureau for Population, Refugees, and Migration. The experience of these evaluations will inform the development of a standardized health evaluation framework for use in responding to humanitarian crises.

Table of Contents

Acknowledgements...... 9

1. Executive Summary...... 1 1.1 The Context...... 1 1.2 UNHCR Health Programme ...... 1 1.3 The Evaluation...... 2 1.4 Main Conclusions...... 2 1.5 Main Recommendations ...... 6 1.6 Main Lessons Learned ...... 8 2. THE CONTEXT...... 11 History ...... 11 The Host Country...... 12 Repatriation ...... 13 The Phenomenon of Transmigration ...... 15 Conclusions...... 16 Recommendations ...... 16 Lesson Learned...... 16 3. UNHCR HEALTH PROGRAMME...... 17

4. THE EVALUATION: PURPOSE, METHODOLOGY AND EVALUATION TEAM...... 19 Findings ...... 19 Conclusion ...... 21 Recommendations ...... 21 Lessons Learned ...... 21 5. FINDINGS...... 23 5.1 Refugees and their health ...... 23 Camp Refugees...... 24 Urban refugees...... 24 Conclusions...... 25 5.2 Health Policy...... 25 Conclusions...... 27 Recommendations...... 27 Lesson Learned...... 27 5.3 Assistance Strategy...... 27 Implementing Partners...... 27 Project Directorate for Health...... 28 Monitoring and Evaluation...... 29 Going Where? ...... 30 Conclusions...... 33 Recommendations...... 33 Lessons Learned ...... 34 5.4 Health Information System...... 34

Data collection and processing set-up...... 35 Comments on the HIS reports ...... 36 Conclusions...... 38 Recommendations...... 39 Lessons Learned ...... 39 5.5 Coordination ...... 40 Conclusion...... 40 Recommendations...... 41 5.6 Norms and standards...... 41 Conclusion...... 41 Recommendations...... 41 Lesson Learned...... 42 5.7 Nutrition...... 42 Food Security ...... 43 Growth Monitoring and Nutrition Surveys ...... 45 Supplementary Feeding ...... 46 Micronutrient Deficiencies...... 46 Conclusions...... 47 Recommendations...... 47 Lessons Learned ...... 48 5.8 EPI and Child Health...... 48 EPI ...... 49 Child Health...... 50 Conclusion...... 50 Recommendations...... 50 Lesson Learned...... 51 5.9 Reproductive Health ...... 51 Conclusions...... 54 Recommendations...... 54 Lessons Learned ...... 54 5.10 Curative services - Clinical Care in the BHUs...... 55 Conclusions...... 56 Recommendations...... 56 Lessons Learned ...... 57 5.11 Medical supplies...... 58 Conclusions...... 59 Recommendations...... 59 Lesson Learned...... 59 5.12 Training ...... 60 PDH and training ...... 61 Budgetary Cuts...... 61 Conclusions...... 62 Recommendations...... 62 Lesson Learned...... 62 5.13 Security...... 62 5.14 Gender-related Issues...... 62 Women’s Access to Health Care ...... 63 Community Health Workers (CHWs)...... 64 Registration ...... 65 Activities for Women...... 65 Adult Education ...... 65 Conclusions...... 66 Recommendations...... 66

5.15 Protection...... 66 Sex and Gender Based Violence (SGBV) ...... 67 Conclusions...... 68 Recommendations...... 68 Lessons Learned ...... 68 5.16 Referral...... 69 Conclusions...... 70 Recommendations...... 70 Lesson Learned...... 70 6. EVALUATION CRITERIA...... 71 6.1 Impact ...... 71 Conclusion...... 72 Lesson Learned...... 72 6.2 Relevance ...... 72 Lesson Learned...... 72 6.3 Appropriateness and Participation ...... 73 Conclusions...... 73 Lesson Learned...... 73 6.4 Effectiveness...... 73 Conclusion...... 74 Recommendation...... 74 6.5 Connectedness and integration...... 74 Conclusion...... 75 Recommendation...... 75 6.6 Coherence...... 75 Conclusion...... 76 Recommendations...... 76 Lesson Learned...... 76 6.7 Coverage...... 76 Conclusion...... 77 Recommendation...... 77 6.8 Efficiency...... 77 Conclusions...... 79 Recommendations...... 79 Lesson Learned...... 79 Annex 1: Bibliography...... 81

Annex 2: Chronology of itinerary, contacts and activities...... 87

Annex 3: Terms of Reference...... 95

Annex 4: List of Implementing Partners for Health...... 111

Annex 5: List of Acronyms...... 113

Annex 6: Questionnaire to Follow Up Pilot Interagency Health Evaluations ...... 115

Acknowledgements

Whilst conducted independently, the field visit made use of the facilities, support and information offered by and gratefully accepted from UNHCR field offices and staff. During the briefing meeting in Geneva and throughout the field trip, essential background information and orientation were readily provided. The evaluation expresses its gratitude to all those —staff, partners, beneficiaries and external actors —who kindly gave their time and contribution. Especially appreciated was the frankness of UNHCR staff.

1. Executive summary

Evaluated Action: Pakistan (Health and Health Programmes for Afghan Refugees)

Date of the Evaluation: 1 - 18 December 2003 (field mission)

Consultants: Markus Michael MD, DTM, MPH, independent consultant

Mary Corbett MSc, SRN, CTCCC, CTCM&H, independent consultant

Glen Mola MB.BS. (Melb), DPH (Syd), FRANZCOG, FRCOG, Professor of Obstetrics & Gynaecology at the University of Papua New Guinea.

1.1 The Context

Several waves of Afghan refugees have flooded into Pakistan since 1979. Between 1979 and 1997, the UNHCR spent more than US$ 1 billion on Afghan refugees in Pakistan.1 Following the decrease of assistance in 1995 by UNHCR and particularly WFP’s decrease in food aid, a mass migration took place of up to one million Afghan refugees to urban centres in Pakistan. New camps for post-9/11 refugees were established, but the bulk of current camp refugees (over 1.1 million) live in the old camps, called ‘refugee villages’. An equal number or more live in urban centres, many of them in slums, and do not benefit from UNHCR’s basic health or education programmes. Altogether, over two million refugees remain in Pakistan in protracted legal limbo,2 and are not liable to return soon; UNHCR’s massive repatriation programme of 2002 has lost steam. Presently, UNHCR in Pakistan is not facing a clear-cut refugee problem, but a much more complex phenomenon of transmigration, ‘in which political, ethnic, economic, environmental and human rights factors combine and lead to population movements’.3

1.2 UNHCR Health Programme

The health care of the Afghan refugees in Pakistan has been the responsibility of UNHCR; the largest proportion of UNHCR’s annual budget is used for this purpose. The programme targets camp refugees; the cost is presently around US$ 1.2 million. The Primary Health Care (PHC) programme is implemented through 117 Basic Health Units (BHUs) operated by 15 Implementing Partners (IPs), the largest of which is the governmental Project Directorate for Health (PDH), which received US$ 2.74 million in 2003. Two IPs support the other IPs and their BHUs with disease- specific programmes for malaria, tuberculosis and Leishmaniasis.

1 Turton D, Marsden P (2002) Taking Refugees for a Ride? The politics of refugee return to Afghanistan. Afghanistan Research and Evaluation Unit. [UNHCR Afghanistan]. 2 Van Hear N (2003) From durable solutions to transnational relations: home and exile among refugee diasporas. UNHCR/EPAU New Issues in Refugee Research. Working Paper No 83. 3 Crisp J (1999b) Policy challenges of the new diasporas: migrant networks and their impact on asylum flows and refugees. UNHCR, New Issues in Refugee Research. Working Paper No. 2.

1 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Secondary health care is also supported by UNHCR through IPs who provide a general referral system to local hospitals some of which also receive some UNHCR funding. UNHCR also has a system in place to refer/transfer patients to institutions outside Pakistan if specialized treatment is required and not available within the Pakistani health system.

1.3 The Evaluation

The evaluation field visit took place between December 1st and 18th, 2003. The evaluators met contacts in Islamabad and visited refugees in camps and urban settings in Islamabad and in the provinces of NWFP and Balochistan. The bulk of the investigation was conducted through semi-structured interviews (with individuals and groups) and document study, complemented by focus groups, direct observation, the administration of a questionnaire to IPs and a rating of IPs by UNHCR field officers. The team generally travelled together, but tended to separate once the designated location was reached, in order to increase the breadth of the evaluation. Independent translators were used whenever the situation required their support.

The Joint UNHCR/WHO evaluation of health and health programmes for Afghan refugees living in Pakistan was part of a three-year initiative to institutionalize interagency health evaluations in humanitarian crises. The Interagency Health Evaluations in Humanitarian Crises Initiative (IHE-HCI), spearheaded by UNHCR and WHO in 2003, has been a combined effort of United Nations agencies and organizations (WHO, UNHCR, UNICEF, and UNFPA), NGOs (Action Against Hunger, Epicentre, ICRC, IFRC, IRC, Merlin, MSF, Oxfam and Save the Children) and academic institutions (Columbia University and London School of Tropical Medicine and Hygiene). This evaluation was the second pilot interagency assessment that has been carried out; others were also conducted in Nepal and Zambia. Lessons learned from the three evaluations will be used to identify best practices, to develop Guidelines for Interagency Health Evaluations, and to establish policy for future sector-wide assessments of humanitarian response in health.

In March 2004 the report was first made available in a pre-publication to serve as a basis for discussion during an Interagency Health Services Evaluation Workshop hosted by UNHCR. In July 2004, a Project Coordinator was appointed to manage IHE-HCI activities, including following up on the outcomes of the three pilot evaluations. Further information on the follow up recommendations to the Pakistan evaluation is provided in Annex 6.

1.4 Main Conclusions

The Context

UNHCR in Pakistan is not facing a clear-cut refugee problem, but a much more complex phenomenon of transmigration. Consequently, traditional instruments and concepts of refugee crisis management are inappropriate in this context.

2 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

The Evaluation

The only major methodological shortcoming of the evaluation was a potentially flawed sampling frame. Some other methodological issues such as missing control groups, difficulties in accessing women and a lack of genuinely ‘representative’ respondents were to a great extent overcome. Refugees and their Health

The level of PHC services for refugees in camps, and most probably their overall health status, is better than that of the host population in Pakistan and people living in Afghanistan. This does not mean that UNHCR is aiming too high, but that standards in the host country and Afghanistan are too low. The single most important finding of the evaluation is the stark contrast between health and health services for camp and urban refugees. For the latter, access to preventative and curative primary health care is difficult or non-existent. Assistance Strategy

IP performance was found to be generally good, with the main exception being the Project Directorate for Health (PDH), In absence of a real ‘exit strategy’, UNHCR is applying an emergency strategy to a problem that has become perennial. Providing externally-funded primary care services, far beyond those available locally, for a population that will not, in the main, be repatriated, is not a sustainable solution. Health Information System

After five years of operations, the national HIS coordinating unit system has yet to provide a reliable or useful compilation or analysis of data for the BHUs NGOs or UNHCR. Most IPs have a reasonable system of collecting data at the BHU level; however, they all rely upon voluntary, poorly-educated Afghani community health workers to collect denominator data. Coordination

Shortcomings with regard to coordination between UNHCR offices, UNHCR and its OPs/IPs, UNHCR and the Pakistan MoH, and the IPs, at times lead to service overlap or gaps. Norms and Standards

Standard protocols for serious problems are largely available but not readily so for the BHU clinicians There is a dearth of standard protocols for the management of minor but commonly presenting ailments. Nutrition

Food security for new camp refugees is more fragile than for the old camp refugee population, a fact that justifies maintaining a regular full general ration.

3 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Global acute malnutrition is within acceptable levels, suggesting that specialized feeding programmes are not needed. Chronic malnutrition in Balochistan camps is a serious issue and needs to be urgently addressed. EPI and Child Health

EPI coverage in the camps is better than that of the host population or for Afghanistan. No major outbreaks of measles have been reported. Reproductive Health

Reproductive Health should be a critical part of any emergency and post- emergency health programme; however, many IPs omit RH (or only engage in the ‘easier’ parts of RH) because they find it a difficult health programme to implement. A number of NGOs have developed some very innovative and successful RH programmes that can be easily extended to other areas. Curative Services

BHU doctors are often overwhelmed by a very large caseload of minor ailments that leads to inefficiencies in the curative health care system. There is virtually no clinical care supervision in the BHUs. It appears that in the absence of other meeting points for women and their children, that the BHU is used. This can be detrimental to the effectiveness of curative care services. Medical supplies

Decentralizing procurement (whereby the tendering process, drug quality control and control of procedures need to be executed three times instead of once), leads to inefficiencies (e.g. UNHCR losing the opportunity to obtain better prices) and opens the way to the possibility of embezzlement. Irregularities in the process of drug and medical supply procurement by PDH may have caused UNHCR a yearly financial loss of about US $200,000, if not more. Training

Some very good, culturally-sensitive, language-appropriate training materials have been developed for the Afghan refugee populations, which can be used by other IPs. There is no need to re-invent the wheel. Coordinating and combining the training of stakeholders is more cost effective. Gender-related issues

Tremendous gender inequality within the Afghan refugee population affects the basic rights of women. Programmes need to be innovative and sensitive in order to address this inequality.

4 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Protection

Health-related protection issues mainly concern Sexual- and Gender-Based Violence (SGBV); they include rape and forced under-age marriage. Referral

Many resources (both in terms of staff and in terms of assistance to the refugees) are used in facilitating referrals to a mostly ineffective secondary and tertiary health care service. Impact

The UNHCR health programme has had a deep and lasting impact on the camp refugee population. Appropriateness and Participation

The health programme has generally been implemented in an appropriate way, taking the customs and practices of the refugee population into account. However, there is a lack of female health staff in certain areas and a degree of ‘cultural over-sensitivity’ regarding RH-related subjects, which can lead to missed opportunities. However, community participation, especially in the material sense, is limited. Effectiveness

The upward trend over time of certain outcome indicators in Balochistan, i.e. the difference observed between new and old camps can be taken as a proxy for the effectiveness of the health programme. Connectedness and Integration

In the long run, of the current PHC programme stakeholders, only the Afghan communities and the MoH will remain. However, neither of these is currently involved in the health programme. Coherence

The assistance strategy employed - funding of mostly foreign or purpose-created IPs with progressive budget cuts – does not appear to be embedded in or supported by an explicit overall policy of UNHCR toward the Afghan refugees. Coverage

While roughly half the Afghan refugees – those based in camps – benefit from reasonably good UNHCR PHC service, the other half – those based in urban areas – receive no health benefits at all. Efficiency

A sample of NGOs – those who do not have co-funding – appear to work more efficiently than the Project Directorate for Health (PDH), especially with regard to their staff and budget per health unit. With only one health coordinator based in Islamabad, UNHCR is understaffed at the managerial level.

5 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

1.5 Main Recommendations

The Context

Adopt a health policy based on the assumption that the bulk of Afghan refugees will remain in Pakistan. The Evaluation

Comprehensive evaluations of costly health programmes need to be carried out periodically or earlier in their lifetimes. Comprehensive evaluations should be organized as ‘staggered’ evaluations. Assistance Strategy

Improve the current unsatisfactory performance of PDH, either through discontinuing cooperation with PDH as an implementing partner, or restructuring/downsizing PDH to a ‘minimal option’, thus improving the status quo. Introduce formal performance monitoring mechanisms and periodic evaluations according to the criteria elaborated on and agreed to by the IPs. Introduce periodic, formal and transparent tendering processes for contracting services to IPs in connection with the monitoring and evaluation process. Move the goalposts in the direction of increased sustainability; new objectives must be formulated at a strategic level, in view of a prolonged, perhaps indefinite, refugee problem. Health Information System

A rectification process needs to begin immediately; either through a project evaluation or a tripartite review of the collaborative health information system (HIS) system by the four interested parties in Islamabad (CDC, IRC, UNHCR and IPs). The local HIS loop of data collection, compilation and local feedback needs to be strengthened. Coordination

Regular structured coordination meetings need to be introduced and conducted. Norms and Standards

Each basic health unit (BHU) should have a small reference library of appropriate clinical materials for health workers to use. UNHCR should assist an NGO in developing a pocket book for the standard management of common illnesses found in Afghan refugee camps. Nutrition

Continue annual nutrition surveys in order to monitor the nutritional status of the refugee population.

6 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Stop the supplementary feeding programme (SFP) when the present food supply is finished, preferably when the winter is over. However, in the meantime, UNHCR should look at reintroducing the recommended standard ration. Address the present high levels of chronic malnutrition in Balochistan with a blended food ration as part of the general ration, particularly in the Chaman new camps. EPI and Child Health

Develop basic standard guidelines for EPI procedures and coverage for all IPs, and to create a simple system for collecting relevant data as part of the HIS. Reproductive Health

A reproductive health technical support unit should be established to assist IPs in setting up and operating reproductive health programmes. There is an urgent need for tested, culturally-sensitive RH community and health worker education programmes, protocols and services. Many good materials have been developed by NGOs that could be used more widely. Afghan males have traditionally made all the decisions in RH matters for their families. RH initiatives for Afghan refugees need to emphasise male education and involvement. Curative Services

IPs need to have a clinical supervisor who can assist in developing a more efficient patient flow at the BHUs, provide clinical supervision of doctors working in the BHUs, and provide some in-service training from time to time. The patient flow process through the BHU should begin with triage or screening of clients for those who are very ill and need priority attention. Training on the rational use of drugs is a priority. Medical supplies

UNHCR should take drug procurement into its own hands and centralize it in Islamabad. Training

Set up a core group, identify the training tools available, and streamline and conduct trainings according to need. Gender-related issues

Actively recruit female staff to vacant health posts and address operational difficulties to their working in rural areas. Improve the ratio of female-to-male community health workers (CHWs). Develop more innovative social programmes, which can be used as a potential outlet for female refugees. At present, BHUs appear to be the main or only place people can meet and socialize.

7 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Protection

Systematically gather and in a multi-sectoral effort, data on SGBV, in order to learn the magnitude of the problems and to explore potential solutions in co- operation with local partners Referral

Commission a local evaluation on the functioning of Chaman Hospital. The findings will be shared with the main stakeholders (including the locally active NGOs) in order to establish a strategy for revitalizing it for as long as the camps exist. Appropriateness and participation

Revive the attempt to increase financial participation through user fees (linked to increased participation in decision-making). Effectiveness

Measure effectiveness of input, process and output indicators (e.g., number of staff, training provided and patients seen) and make an effort to systematically measure the health outcomes, such as treatment success, behavioural change and knowledge gains. Connectedness and Integration

Discuss the current and future impact of budget cuts in a transparent way with IPs and donors, and look at alternative assistance strategies that take a longer timeframe into account. Include the MoH in policy considerations and as a partner in the planning process. Coverage

Include the urban refugees into UNHCR’s health policy framework. Efficiency

Strengthen UNHCR’s technical/managerial capacity by filling the position in Peshawar with a senior public health professional, preferably an expatriate with a three-year mandate. A focal point for health is also needed in Quetta. Improve efficiency in various areas, as suggested in Chapter 6.8 and include efficiency indicators in routine monitoring and evaluation.

1.6 Main Lessons Learned

The Context

The solution for the refugee problem lies in guaranteeing human rights and the possibility of economic survival in Afghanistan. If UNHCR makes policy decisions without this (but simply assumes the arrival of peace and prosperity in Afghanistan), it could have disastrous consequences for camp refugees.

8 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

The Evaluation

The formative value of a unique comprehensive evaluation late in the lifetime of a programme can be compromised by the – probably long-lasting – argument ‘we’re winding down.’ Health Policy

If UNHCR’s health policy is resource-, instead of needs-based and conforms to a definition of refugee status imposed by the host country, UNHCR risks missing half the population, including its most vulnerable clients. Assistance Strategy

UNHCR will not be able to keep on cutting its budget, while expecting that its health objectives will remain unchanged. Local NGOs originating from or rooted in the refugee communities may be given the capacity to provide services in a more sustainable way, through a level of cost-recovery that puts prices still below the private market, but makes a significant contribution to the provision of services. Health Information System

A top down approach to HIS, – no matter how sophisticated the software – is unlikely to produce useful results and analysis unless it is supported by data collection that can be validated. Analysis of HIS data requires more than expertise in biostatistics and epidemiology. Nutrition

When the SFP programme provides dry take-home food, families share the foods. The ration needs to be increased in order to ensure that beneficiaries recover from moderate malnutrition. Emergency feeding programmes such as SFP and TFP do not address chronic malnutrition. Reproductive Health

To reduce the maternal mortality rate (MMR) and prevent dramatic accelerations in population in the refugee camps, RH education for both men and women should be introduced, and RH services provided at the onset of emergency operations. Cultural over-sensitivity has led to some IPs being ineffective (or indeed not doing much at all) in the RH area. Curative Services

Expecting any doctor to see more than 100 patients in five hours will lead to superficial clinical assessments, occasional misdiagnosis and a failure to notice the real problems of some seriously ill patients. Small user fees can serve to prevent ‘social over-usage’ of health services in the refugee camps.

9 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Training

Budget cuts often lead to substantial reduction in training, which is a false economy. Gender-related issues

Budgets cuts often result in the termination of social programmes such as adult literacy and skills training. The loss of these programmes has a negative impact on health, especially for women. Impact

Distributional effects can significantly tarnish the achievements of an otherwise successful programme. For example, urban refugees have been excluded. Relevance

With regard to important changes over time in the refugee situation (urban migration and 'permanence' of the camp situation), a ‘process approach’ is indicated, rather than a continuation of the initial ‘blueprint approach.’ Appropriateness and Participation

‘Cultural over-sensitivity’ with regard to RH-related subjects can lead to missed opportunities (e.g. tackling the issue of RH with young girls and men). Coherence

Decreases in service provision are only marginally effective as an incentive for voluntary repatriation. Efficiency

‘What is not monitored cannot be evaluated’, and therefore the monitoring of efficiency indicators such as unit cost (e.g. per BHU, per consultation) is an important part of the evaluation of IP performance.

10

2. THE CONTEXT

History

1. Several waves of Afghan refugees have flooded into Pakistan and Iran since 1979 in numbers of comparable magnitude.4 Owing to different refugee policies of the host countries and for geopolitical military reasons – refugee camps in Pakistan were used as ‘strategic sanctuaries’,5 i.e. military training camps for mujaheddin factions – UNHCR’s response in both countries has been quite different. This has been reflected in UNHCR budgets, ‘between 1979 and 1997, UNHCR spent more than US$ 1 billion on Afghan refugees in Pakistan, but only US$ 150 million on those in Iran’.6 To a certain degree, because of this, camp refugees in Pakistan have become dependent on UNHCR.

2. After the fall of the Najibullah regime in 1992, UNHCR and WFP, ceding to donor pressure7 and against the evidence of continuous warfare in Afghanistan, decreased assistance to camp refugees. This was most dramatic in the case of WFP, which stopped food assistance altogether. The withdrawal of assistance resulted in a mass migration of up to one million Afghan refugees into urban centres in Pakistan. The magnitude of this phenomenon was quite unforeseen.

3. New camps for post-“9/11” refugees have been established. Some of these have been already closed, following the repatriation and relocation of their inhabitants. The bulk of current camp refugees (over 1.1 million) live in the old camps, also called ‘refugee villages’; the majority (75%) originates from the eight (Pashto) border provinces in Afghanistan. Relocation, rationalization and closure of (new) camps are current practices, which prove difficult in areas where border camp labour is part of a cross-border economy (including poppy growing).

4. Reports of police harassment were made by refugees and were repeated in the literature, ‘mistreatment at the hands of Pakistani… law enforcement authorities’.8 Because of their undocumented status, refugees are frequently subject to such treatment. 'Old’ refugees therefore do not ask for assistance anymore, but rather for identity papers. A number of refugees do possess them. A Pakistani doctor related, ‘I’m a Pakistani, I have one ID; THEY have three! The recent introduction of a modern ID model, however, makes faking IDs more difficult.

5. In order to understand the impact of UNHCR’s health policy on refugees, a rough categorization based on attitude to possible repatriation and current living conditions of the refugees is presented in Table 1. It is important to keep in mind that the bulk of refugees belong to categories 1 and 3, and are unlikely to repatriate.

4 In 2002, there were 3.5 million Afghan refugees in Pakistan and 2.3 million in the Islamic Republic of Iran. 5 Jean, F. and Rufin, J.-C. (eds.) (1996) Economies des guerres civiles. Hachette, Paris. 6 Ibid. 7 Ibid. 8 Closed Door Policy. Afghani Refugees in Pakistan and Iran. Human Right Watch Report, Vol. 14 No. 2 (G), February 2002.

11 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Table 1: Categories of Refugees Category Conditions Attitude to repatriation 1. Old camps Economically integrated (no subsistence Unlikely to repatriate, unless a (Refugee aid) ‘miracle’ happens in Afghanistan, villages) Some move freely within Pakistan & or if forced backwards and forwards to Afghanistan ‘Wise choice’ and to the Middle East primary health/ education by UNHCR 2. New camps Full basic UNHCR support Refugees from the north of Afghanistan may repatriate, with ‘Mixed bag’ exception of Pashto (security concerns) 3. Urban poor Economic survival as unskilled labour Unlikely to repatriate, unless a Mostly uneducated miracle happens in Afghanistan, or ‘Negative Have neither capital nor land in if forced selection’ Afghanistan No basic services from UNHCR 4. Urban Have business, employment Educated Will not return, unless forced middle class Have capital and Pakistani ID

‘Can make Have family overseas and foothold in choices’ Afghanistan

The Host Country

6. Afghan refugees in Pakistan were never registered as such by the Government of Pakistan (GoP); prima facie recognition was granted until the late 1990s, when this policy was stopped. 9 Pakistan is a poor country with a series of problems of its own. In 2001, the population was 140.5 million, the GNP per capita only US$ 429, and an adult literacy (in 1998) of only 42.7%.10 Pakistan is not a signatory to the 1951 Refugee Convention.

7. Pakistani politics and the Afghan conflict are interlinked. A rise of religious parties has been observed in Pakistan during the last decade. Observations in urban refugee settings in Quetta indicated that the breeding grounds that produced the Taliban a decade ago are as fertile as ever. Afghan refugee boys in the slums are more likely to frequent madrasas rather than schools. In certain instances, refugees are used as pawns for local politics, such as in Balochistan, where there are ethnic tensions between Balochis and Pashto.11 Pashto politicians are said to have an interest in keeping (Pashto) refugees as voters.

8. Although the GoP recognizes that ‘voluntary repatriation, where feasible, constitutes the preferred durable solution…’, the spectre of involuntary repatriation casts its shadow over Afghan refugees. Involuntary repatriation is today generally ‘coming to be pursued as a solution to the refugee problem because in the post-Cold War era the rich northern states see no reason to share the burden of the poor south

9 Jean, F. and Rufin, J.-C. (eds.), 1996, op cit. 10 WHO emergency response in the fallout of the Afghan crisis in Pakistan, WHO Country Office, Pakistan, Islamabad, April 2003. 11 Haleem I (2003) Ethnic and sectarian violence and the propensity towards pretorianism in Pakistan. Third World Quarterly, 24 (3): 463-477.

12 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 at both the level of asylum and resources’.12 In practice, however, it is difficult to imagine repatriation being carried out by force and on a large scale. The GoP is explicitly against “Development Through Local Integration”, which is claimed to be ‘not an option’.13 Understandable is the utterance of a GoP official, ‘if donors have donor fatigue… then we have asylum fatigue…’.14

9. At present, the GoP is pushing for registration of all Afghans in Pakistan as refugees. The Chief Census Commissioner has made proposal for a census at an estimated cost of US$ 2 million.15 UNHCR does not appear too keen on this census, possibly because it would significantly increase its caseload. The general utility of gathering baseline data appears justified for both host government and UNHCR alike. But refugee statistics ‘will always be a source of controversy and dispute’,16 whilst at the same time ‘without registration, refugees have no rights’.17 Others argue, in the light of the potential for involuntary repatriation, that ‘screening could have a drastic effect on the lives of undocumented Afghans’.18

Repatriation

10. UNHCR conducted a massive repatriation programme in early 2002 and registered 1.56 million returnees from Pakistan, not counting the spontaneous (unassisted) returns.19 Owing to insufficient conditions for resettlement in Afghanistan, this was a dubious success, and ‘arguably, in the interests neither of the majority of its intended beneficiaries nor of the long-term reconstruction of Afghanistan’. In assisting a mass return of refugees to Afghanistan in 2002, UNHCR as responding more to the perceived political interests of its donors and host governments, than to the actual interests of the majority of its “beneficiaries”.20 It is not surprising that only 341,732 returned instead of the 600,000 expected for 2003.21

11. The UNHCR temporarily closed repatriation centres in Peshawar and Quetta due to the killing of a UNHCR officer in Ghazni on 16 November.

12. The repatriation process had its hitches, including an ‘unknown number of “recyclers”; seasonal migrants, and [an] unquantifiable “backflow”.22 The latter is substantiated by anecdotal evidence; a razed refugee slum in Peshawar has recently been settled in once again.

12 Chimni B (1999) From Resettlement to Involuntary Repatriation: towards a critical history of durable solutions to refugee problems. Evaluation of Humanitarian Assistance in Emergency Situations. UNHCR/EPAU New Issues in Refugee Research. Working Paper No 2 [UNHCR durable solutions]. 13 Agenda of Tripartite Commission Meeting in Dubai, 3.12.2003, chaired by UNHCR. 14 Ibid. 15 Ibid. 16 Crisp J (1999a) Who has counted the refugees? UNHCR and the Politics of Numbers. New Issues in Refugee Research. Working Paper No. 12 [UNHCR counting refugees]. 17 MSF press release quoted in Crisp J., 1999a, ibid. 18 Ibid. 19 Estimated by UNHCR at 40,000 through Chaman and Turkham. 20 Turton D, Marsden P (2002) Taking Refugees for a Ride? The politics of refugee return to Afghanistan. Afghanistan Research and Evaluation Unit. [UNHCR Afghanistan]. 21 Agenda of Tripartite Commission Meeting in Dubai, 3.12.2003, chaired by UNHCR. 22 Turton D, Marsden P (2002), op. cit.

13 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

13. Most returnees who took advantage of the recent repatriation exercise came from urban areas, and less than 10% of the camp populations have since returned to their home countries. Urban refugees, ‘those who were having difficulties making ends meet would have been most likely to put their trust in the “encouraging messages” they were receiving from the international community’.23 Many who repatriated were refugees from the year 2000 who had fled fighting and drought in central and northern Afghanistan, or were part of the urban populations that had either fled the 1992-1995 fighting in Kabul or, between 1996 and 2000, who sought refuge from the Taliban. The camp dwellers (termed the ‘wiser caseload’ by an official from the Chief Commissionerate for Afghan Refugees (CCAR), however, remained in the camps, comparing the conditions there favourably with Afghanistan. As mentioned above, the overwhelming remaining refugee population originates from the Eastern and Southern bordering provinces, where ‘there is no development, reconstruction or rehabilitation work taking place at the moment, due to security concerns’.24 These refugees are not liable to return - if at all - until there are significant improvements in Afghanistan or in the unlikely event of forced repatriation.

14. The ‘party line’ given by refugee village elders is that they cannot return because of ‘lack of security’: harassment by Kabul government because they are suspected to be Taliban/Al Qaida members, or because they are fearful of being at the mercy of warlords. Others will not return ‘as long as the Americans occupy Afghanistan’. However, refugee women, mention economic factors, ‘we can survive here, but not in Afghanistan’ as one poor widow in a camp-like settlement put it.

15. Despite concerns expressed about returning to Afghanistan, focus group interviews revealed that many male refugees are travelling back and forth to Afghanistan on a regular basis - some as often as monthly or twice monthly.

16. While some may have legitimate ‘ethnically based fears associated with reprisals’,25 the validity of the assumption that ’everyone wants to return to the country of origin, i.e. “home”’ is not questioned often enough, at least not officially’. A crucial factor is the passage of time, ‘second generation refugees may not want to return to a home they know little about’.26 These days, there are third-generation Afghans being born in Pakistan! Changes in lifestyle may be irreversible. After nearly 20 years in Pakistan, former farmers have become ‘used to urban conditions and occupation or the ‘refugee lifestyle’’27 and some kuchis (nomads) may never migrate again.

17. The prediction made in 2002 - ‘it is likely that around 2 million Afghans will remain in Pakistan... after the 2002 “repatriation - season” has ended’28 – seems to have come true. This has produced an apparent deadlock - ‘the GoP will never accept them as migrants, the refugees will never go back. The solution is unclear, as

23 Turton D, Marsden P (2002), op. cit. 24 Agenda of Tripartite Commission Meeting in Dubai, 3.12.2003, chaired by UNHCR. 25 Human Right Watch Report (2002), op. cit. 26 Chimni B (1999), op. cit. 27 Turton D, Marsden P (2002), op. cit. 28 Ibid.

14 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 relocation in Pakistan or elsewhere does not appear to be a realistic option. Will the refugees remain forever in ‘protracted limbo’?29

18. A potential solution for the refugee problem lies in Afghanistan, with guarantees for human rights and possibilities for economic survival. How – and whether - Afghanistan will make a miraculous transition from tribal system and warlordism to democracy is at present unclear. Despite the fact that at the time of report writing, the Loya Jirga in Kabul has accepted a new constitution and elections remain scheduled for summer 2004, Afghanistan ‘remains a fractured, undemocratic collection of “chiefdoms” in which warlords are free to intimidate, extort and repress local populations…’.30 No wonder the families of most of Afghanistan’s current political leaders live in Pakistan. It is clear and recognized31 that UNHCR cannot address the structural and economical problems of Afghanistan.

The Phenomenon of Transmigration

19. The fate of Afghan refugees in Pakistan demonstrates that ‘refugees are part of a complex migratory phenomenon, in which political, ethnic, economic, environmental and human rights factors combine and lead to population movements’.32 The Afghan refugee “crisis” of the past 20 years has been overlaid ‘on a history of economic migration… going back hundreds of years’.33 ‘Transnationalism’, which includes split families and seasonal migration has been considered ‘an “enduring” if not a “durable” solution to displacement’.34 The categorization of different kinds of forced migrants has been called illusory in contexts where one individual may go through different stages – refugee, migrant labourer, asylum seeker – and where one extended family is composed of members of different categories.35

20. UNHCR in Pakistan is therefore not facing a clear-cut refugee problem, but a much more complex phenomenon of transmigration. ‘The problem of how to regularize, in a globalized world, the clandestine movement of people, desperate to escape from the “discomforts of local existence” may well turn out to be a hallmark of the present century, just as the “refugee problem” was a hallmark of the last’.36

29 Van Hear N (2003) From durable solutions to transnational relations: home and exile among refugee diasporas. UNHCR/EPAU New Issues in Refugee Research. Working Paper No. 83. 30 All Our Hopes are Crushed. Violence and Repression in Western Afghanistan. Human Right Watch Report, Vol. 14 No. 7 (C), November 2002. 31 Chimni B (1999), op. cit. 32 Crisp J (1999b) Policy challenges of the new diasporas: migrant networks and their impact on asylum flows and refugees. UNHCR, New Issues in Refugee Research. Working Paper No. 2. 33 Turton D, Marsden P (2002), op. cit. 34 Van Hear N (2003), op. cit. 35 Ibid. 36 Turton D, Marsden P (2002), op. cit.

15 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Conclusions

21. The unilateral UNHCR/WFP decision of 1992 to decrease assistance drastically by 1995, and the lack of subsequent adjustment to realities in Afghanistan by 1995, resulted in a mass exodus of refugees into urban settlements, where they were much worse off thereafter.

22. The only window of opportunity for mass repatriation was the summer of 2002. The bulk of remaining Afghan refugees in Pakistan are unlikely to repatriate, unless a security and economic miracle happens in Afghanistan, or unless they are forced to go.

23. The breeding grounds that produced the Taliban a decade ago in poor Afghan refugee slums are as fertile as ever.

24. UNHCR in Pakistan is not facing a clear-cut refugee problem, but a much more complex phenomenon of transmigration. Traditional instruments and concepts are ill equipped to deal with it.

Recommendations

25. Put a caveat on proactive UNHCR policies that may be driven by donors and the host country, in disregard of realities in Afghanistan. A more reactive attitude to the latter is indicated.

26. Adopt a health policy that is based on the assumption that the bulk of Afghan refugees will remain in Pakistan.

Lesson Learned

27. The solution for the refugee problem lies in Afghanistan, with guarantees for human rights and possibilities for economic survival. If the UNHCR makes policy decisions based merely on an assumption of peace and prosperity in Afghanistan, it could have disastrous consequences for camp refugees.

16

3. UNHCR HEALTH PROGRAMME

28. The health care of the Afghan refugees in Pakistan is the responsibility of UNHCR. As health care is relatively expensive, the largest proportion of UNHCR’s annual budget is used for this purpose. In 2003 alone, the health budget was over US$ 5.85 million (resources were substantially increased in 2001 with the new influx of refugees). This health care support takes the form of funding for primary and secondary health care for refugees. It is only targeted at the camp refugees, presently around 1.2 million —a number that had fallen from 1.6 million at the start of 2003. The ‘invisible’ refugee load in urban areas receives no health care support. This means UNHCR spends around US$ 3-4 per refugee on health care, which is very low.37

29. Operational Partners (OPs) and Implementing Partners (IPs) carry out the operational hands-on primary health care with funding from UNHCR (see Annex 4). This is done through the Basic Health Unit (BHU), which normally supports a population of around ten to fifteen thousand refugees. UNHCR has a technical health person (who acts as a health co-ordinator) based in Islamabad and two health positions in the field, one in Peshawar and one in Quetta. Both of those positions were vacant during this evaluation. There are presently 117 BHUs supported by 15 IPs. Out of the 15 IPs PDH is the largest, supporting 51 BHUs (about 43% of the total PHC facilities). Seven of the IPs receive co-funding, while two are self-funded. PDH received 47% of UNHCR’s total health budget of 2.74 million dollars. Primary Health Care includes health promotion, EPI, MCH, and curative care. There are a number of labour rooms run by IPs to assist in safe deliveries, but obstetric emergencies are transferred to the nearest tertiary care hospital. Some BHUs have ambulance facilities for these emergency cases while in other areas patients use private transport.

30. Secondary health care is supported by UNHCR in the form of referral assistance to the nearest local hospital; some of these hospitals receive token funding from UNHCR. There are exceptions. In Peshawar, Afghan refugees are mainly looked after by a number of self-funded charity/private hospitals. In Chaman, Balochistan, the local hospital has received substantial funding from IPs and OPs to upgrade the facilities there for both refugees and the local population. UNHCR has also a system in place to refer/transfer patients to institutions outside Pakistan if specialised treatment is required and not available within the Pakistani health system. UNHCR covers these expenses too.

31. Some IPs such as HealthNet International (HNI) and the Association for Community Development (ACD) have taken the lead in specific diseases and support all BHUs and IPs with technical support and training for tuberculosis (ACD), Leishmaniasis and malaria (HNI).

37 At the level of the Basic Package for Health Services currently implemented in Afghanistan. The World Bank, however, pitches it at US $ 12 / year (World Bank (1993) World Development Report 1993: Investing in Health. New York, Oxford University Press.)

17

4. THE EVALUATION: PURPOSE, METHODOLOGY AND EVALUATION TEAM

32. The Joint Evaluation of Health and Health Programmes for Afghan Refugees, for which UNHCR has taken the lead, was developed with WHO (HAC) as part of a series of evaluations38 implemented according to guidelines established in an Interagency Concept Paper.39

33. The purpose of the evaluation, as outlined in the Terms of Reference (Annex 3), was two-fold: (a) to provide operational support for this long-standing humanitarian action; and (b) to contribute to institutional learning by testing the evaluation framework. The evaluation broadly analysed UNHCR’s health programmes in Pakistan for the last two decades. Special attention was paid to the current and future consequences of the ongoing scaling-down of UNHCR’s budget and the potential ways to offset it. The team was made aware that funds would be available for a technical follow-up of the evaluation until the end of March 2004, if need be. No comprehensive evaluation of UNHCR health programmes in Pakistan had been undertaken since the 1980s. Consequently, this report does not include the traditional chapter on ‘follow-up on earlier recommendations’.

34. The evaluation was carried out in four standard phases: (i) briefing in Geneva; (ii) field mission to Pakistan (1-18 December 2003); (iii) debriefing in Geneva; and (iv) preparation and submission of the draft report by end January 2004. Annex 2 lists displacements and contacts made. The team was made of three consultants: Dr Markus Michael (public health physician), Ms Mary Corbett (nutritionist) and Professor Glen Mola (reproductive health specialist with a special interest in health information systems). The report is the result of teamwork, with contributions according to individual expertise.

35. The bulk of the investigation was conducted through semi-structured interviews (with individuals and groups) and a desk study (Annex 1), complemented by focus group discussions, direct observation, the administration of a questionnaire to Implementing Partners (IPs) and a rating exercise of IPs by UNHCR field officers. The team generally travelled together, but tended to fan out once a location was reached, in order to increase the breadth of the evaluation. Wherever the situation required, independent translators were used.

Findings

36. The team’s initial criticism that the TOR presented an un-prioritized, excessive list of issues to be covered was readily admitted during the briefing. The consultants were given discretion to prioritize issues themselves in the field, according to their feasibility and perceived importance. The matter was resolved by giving priority to breadth over depth.

37. UNHCR headquarters and offices in Islamabad, Peshawar and Quetta provided impeccable logistical and other support, and staff readily made time

38 Two others were carried out in Nepal and Zambia during the last quarter of 2003 39 Concept Paper. Establishing Inter-agency Health Programme Evaluations in Humanitarian Crises. Inter-Agency Health Programme Evaluation. Concept Paper draft. 2003.

19 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 available for the team. Ample documentation was provided during briefing and during the field trip.

38. The role of WHO in the evaluation remained unclear during its course. The team was initially made aware that WHO had come on board late owing mainly to communication problems (through WHO Eastern Mediterranean Regional Office in Cairo). WHO field staff accompanied the team in the field for part of the time the evaluation progressed; however, they had no other function than being key informants on general topics.

39. The team had very little influence over the evaluation schedule. In part, this was due to constraints of notification and security. This created obstacles at three levels: first, time was lost with initial formal plenary meetings and secondly, potential sampling biases were introduced. The choices of sites to visit were entirely pre-arranged, which, in conjunction with the high number of potential sites to visit and the already short duration of the evaluation40 resulted in potentially flawed sampling. Without accusing field staff of choosing showcase projects, there is no way to prove that this is not the case; random visits to un-scheduled sites were only possible occasionally. Thirdly, the fact that the visit of the team was announced beforehand everywhere created a certain ‘police car effect’. The team complemented the schedule by visiting sites of concentration of urban refugees in the three cities visited.

40. Other methodological difficulties encountered were related to controls and ‘representativeness’ of key respondents. The problem of ‘missing control groups’ when investigating refugee populations is well described in the literature41 and concerns the local host population and the non-displaced. A foray into Afghanistan was admittedly impossible to organize, and time constraints did not allow for visits to local settings. The evaluation had therefore to rely on previous personal knowledge (with regard to Afghanistan), local informants (with regard to Pakistan) and on available literature.

41. The problem of access to randomly chosen informants is well known, ‘those who come forward or agree to be interviewed may not be representative at all’.42 In an extremely hierarchical society, it required considerable skills to get beyond the officials, i.e. views of the village elders or the most senior person within the group’43.

42. Although the team members were contacted well in advance about their potential participation in the evaluation, two of the consultants were given only one

40 Unintended: squeezed between Eid in Pakistan and Christmas in Switzerland, the field trip had to be cut short by 2-3 days. 41 Jacobsen K, Landau L (2003) Researching refugees: some methodological and ethical considerations in social science and forced migration. UNHCR Evaluation and Policy Analysis Unit, Working Paper No. 90 [Researching Refugees]. 42 Jacobsen K, Landau L (2003), op. cit. 43 This is illustrated by the first, well-prepared, visit to a refugee camp. The evaluators were seated, beside a UNHCR field officer in camouflage jacket and with a Pakistani policeman behind their shoulders, in front of a semi-circle of more than a dozen village elders – some of them wielding mobile phones. Only one of them spoke, however, introducing each sentence with ‘our Pakistani brothers have been very generous…’ A subsequent request for a random household visit was turned down (probably because the evaluator still had the policeman at his back), and the visit to the camp ended finally with collective interview with youngsters at a shop. The evaluators soon managed to significantly increase effectiveness of further visits.

20 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 working day’s notice before departure from home (the third team member received four working days’ notice). The team composition, however, was extremely complementary. The only exception was the gender composition, which was a concern in a context where access to women respondents by male investigators is notoriously difficult, and where additional women on the team would have been desirable.

Conclusion

The only major methodological shortcoming of the evaluation was potentially flawed sampling; others, such as missing control groups, and difficulties of access to women as well as to genuinely ‘representative’ respondents were to a great extent overcome.

Recommendations

Allow, considering the variety of both topics to cover and refugee settings to visit, for a slightly longer evaluation period in the field than two-and-half weeks. Give earlier notice to the consultants than — as it was the case for two of them — one working day accorded between confirmation of contract and departure from home. Tilt, (in a context where access to women respondents is notoriously difficult for male investigators), the gender balance of the evaluation team toward the female component. Carry out such comprehensive evaluations of such costly health programmes earlier in their lifetime or rather, periodically. Organize such comprehensive evaluations as ‘staggered’ evaluations. A previous or earlier field visit by the team leader could: Shape the schedule with a view towards more efficiency; Gather written data sources and write the narrative of the action; Improve sampling and design a methodology appropriate to specific constraints; Consider hiring a local consultant for technical topics; and Include scheduled writing-up time at the end of fieldwork for report.

Lessons Learned

It is important to review forthcoming public holidays in the early stages of planning the evaluation. This can shorten the field mission and thereby compromise thereby its effectiveness and efficiency. The formative value of a unique comprehensive evaluation late in the lifetime of a programme can be compromised by the – probably long-lasting – argument ‘we’re winding down’ (BPRM representative in Geneva).

21

5. FINDINGS

5.1 Refugees and their health

43. In order to understand the health situation of the refugees, it is important to situate it in the Pakistani context. In 2001, WHO estimated that Pakistan spent 4.0% of its GDP on health, which is lower than neighbouring countries India (5.2%) and Iran (4.4%). Local sources, however, including government officials, invariably give figures of below 1%. A general practitioner in a government BHU said, ‘there is no public health system in this country’. A BHU in Quetta allegedly cared for a population of 200,000, but was given a monthly budget of only US$ 50 for drugs. It is no wonder that 80% of the Pakistani population are said to use the private health sector, which appears to be unregulated. In Pakistan, the provision of public health care, including funding is a provincial function, delegated to the Department of Health within the respective provinces. Only certain (vertical) programmes are federally managed and funded, like EPI, TB, malaria, etc.

44. Camp refugees benefit from PHC services, provided by UNHCR and NGOs (see Chapter 3). Some camp refugees, however, make ready use of the private sector, ‘our BHU is fine, but I don’t want to waste time waiting, so I prefer to spend 100 Rs to see a doctor in the bazaar.’ With regard to second- and third-line services, both camp and urban refugees use NGO facilities – where available – and government hospitals. In Peshawar and Balochistan, MoH officials estimate that 70-80% of Afghans are in-patients of tertiary hospitals. Surveys of MoH hospitals performed during the evaluation confirmed a percentage of more than 50%. In Peshawar, charity hospitals still exist and are preferred by the refugees, although they are not free - a complicated delivery may cost the equivalent of US$ 50 (as compared to US$ 100 at a private facility).

45. The obvious link between educational standards and health is exemplified in the lack of hygiene (waste disposal, use of latrines) reported and observed in new camps in Balochistan and in urban refugee slums. Both groups had little or no primary education. Primary schools are available in camps, but secondary school is not available. Evidence indicates that some of the boys in the camps were able to get into Pakistani secondary schools.

46. The single most important finding of the evaluation is the stark difference between health and health services for camp and urban refugees. UNHCR equals de facto ‘refugee’ with ‘camp refugee’44, for which it uses a working numbers of 1.55 m (Annex 3).45 The number of urban refugees is at least the same.

44 The subtle change of the target group definition is visible in a recent UNHCR document, between Goal 3: ‘Refugees who… reside in refugee villages have access to community based health care, primary education and water’; and an ensuing specific objective: ‘Refugees have access to primary health care, in the form of integrated preventive, promotional and curative care’ (UNHCR Pakistan. Achieving our Objectives through Enhanced Partnerships. October 2003), emphasis added.

45 77.4% of which in NWFP, 19.9% in Balochistan and 2.57 % in Punjab

23 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Camp Refugees

47. Housing and shelter conditions in the camps visited — including water and sanitation — appeared satisfactory. There were certain exceptions in the (new) Chaman camps, as not all of the tarpaulin-covered shelters were winter-proof. Moreover, UNHCR’s coal deliveries for heating were delayed. Though the temperature was below zero, neither the women refugees interviewed nor young children had shoes. Circumstances admittedly do not favour investment in these transitory camps. UNHCR wishes to relocate the refugees who will not repatriate in the next spring season, though local authorities have hindered such a move.

48. Camp refugees have become much more health-conscious and more demanding than when they arrived in Pakistan. Because of this health-conscious behaviour and near-universal primary schooling, preventative and curative PHC services are being offered. It is not surprising that the level of primary health care services and the health status of camp refugees are better than either the host country or Afghanistan – a statement respondents unanimously endorsed and which was substantiated by some of the key indicators presented in Table 2. 46

Table 2: Demographic indicators in the Afghan refugee study population with comparable indicators for Afghanistan and Pakistan Indicator Afghan [camp] Afghanistan Pakistan refugees Crude mortality rate 5.5 (5.2-5.8) 19 11 (Per 1000 pop.) Maternal mortality ratio (per 100’000 live 291 (181-400) 820 200 births) Neonatal mortality rate (per 1’000 live births) 25 (22-28) 121 NA Natural rate of pop. Increase (%) 3.7 (2.8-4.6) 2.5 2.8

Urban refugees

49. In absence of a census, the estimate of the number of refugees in urban areas varies between 0.5 and 2.5 million, with a realistic estimate of over 1 million.47 ‘Today [2002] all that is known is that the majority of the Afghan refugee population lives in urban areas’.48 Many of these refugees are the result of a negative selection. After stopping the WFP food assistance to camps in 1995, the ones who managed to earn a living (‘we had to get out of our laziness’) stayed on, and the ones who could not, moved to the cities (as far away as Karachi where up to half a million live) to live in slums. The refugee cards they were holding expired in 1998; they are no longer recognized as refugees by the GoP, which leaves them in legal limbo and subject to police harassment with frequent ‘bailing out’ (respondents’ testimony and different sources).49,50 Most manage to survive as daily labourers or in unskilled occupations.

46 Simplified from 'Maternal Mortality among Afghan refugees in Pakistan, 1999-2000', Bartlett et al, Lancet, 23.2.2003, Vol. 359 643-649. A most useful independent study on the area in Hangu that was visited during the evaluation. 47 Exact figures are given for Balochistan only: 378’000 in camps (251’000 in 11 old ones, 127’000 in six new ones) and 300’000 urban refugees. 48 UNHCR (2002) UNHCR Policy on Refugee Women and Guidelines on Their Protection: An Assessment on Ten Years of Implementation. An independent assessment by the Women’s Commission for Refugee Women and Children. UNHCR Geneva [UNHCR women]. 49 Refugee Reports, Vol. 22, No 7 (2001), by Hiram Ruiz.

24 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Access to primary school is rare; children commonly work as garbage collectors (observation and research).51

50. Housing and living standards are often poor, if not appalling. The evaluators visited long-standing slums, where refugees lived in makeshift dwellings after more than eight years in Pakistan. In a slum in Quetta, three-year-olds went without shoes on the frozen ground. Based on our observations it appears that acute respiratory infections and skin diseases were frequent. The behaviour of children in these situations also shows that the social fabric is being visibly destroyed52.

51. Access to curative primary health care is difficult, as many are too poor to afford private medicine. In Quetta, PRCS is running BHUs for urban refugees and Mercy Corps is running seven, all without UNHCR funding. The situation in Peshawar, where refugee-specific charity hospitals exist, is slightly better than in Islamabad and, most probably, Karachi. In Karachi, if public BHUs are available to them the urban refugees use them. However, owing to the scarcity of public BHUs in Pakistan53, they mainly use the outpatient departments of tertiary hospitals, where they do not have access problems, but have to pay for any drugs prescribed. Preventative care is limited to the national polio immunization campaigns.

Conclusions

The level of PHC services for refugees, and most probably their health status, is better than that of the host population and that of the people of Afghanistan. This does not mean that UNHCR is aiming too high, only that the standards of the host country and that of Afghanistan are very low.54 The single most important finding of the evaluation is the stark contrast between health and health services for camp and urban refugees. For the latter, housing and living standards are often poor, if not appalling, and primary school attendance is rare — children commonly work as garbage collectors. Access to curative primary health care is difficult and preventative care is limited to national polio immunization campaigns.

5.2 Health Policy

52. Two main policy issues have arisen, the above-mentioned definition of target groups — camp vs. urban refugees — and the way UNHCR is dealing with its de facto target group, the camp refugees.

50 Fending for themselves: Afghan Refugee Children and Adolescents working in Urban Pakistan. Women’s Commission for Refugee Women and Children, Mission to Pakistan, January 2002. www.womenscommission.org 51 Ibid. 52 In one instance, the evaluator and his translator were stoned out of neighbourhood 53 Two BHUs visited in Quetta (one of the PRCS and one of the MoH), reported about 80% of Afghan patients in their attendance books. 54 There is an obvious contradiction in the way UNHCR sets its own institutional objectives. The recommendation that ‘services provided for refugees should be at least equal to that of host country nationals – i.e. there must be parity’ (original emphasis). This is not always compatible with their other objectives: ‘health services must be of a quality that… comply with nationally and internationally accepted health standards and principles of medical ethics’ (UNHCR, 2000, Handbook for Emergencies. Second Edition, UNHCR Geneva, added emphasis).

25 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

53. First, who is considered a refugee? The 1951 refugee convention defines them as ‘people who are outside of their country and unable to return to it because they have a “well-founded fear of persecution”’. Considering the fact that the GoP is not a signatory to the convention, this de iure definition may even be irrelevant. Since 1951, however, the definition of ‘refugee’ has been broadened (through the Organisation of African Unity Refugee Convention and the Cartagena Declaration), and now includes ‘people who have sought refuge in other countries as a result of aggression, occupation, generalized violence and events seriously disturbing public order’. It appears that de facto, UNHCR in Pakistan adheres to this wider definition, considering that it extends protection activities to urban refugees as well.

54. Why are urban refugees, who are more vulnerable than camp refugees, excluded from UNHCR health and education services?55 UNHCR staff members unanimously sited budget constraints as the primary reason: ‘we don’t even have enough funds to do what we are supposed to do’. Another reason sometimes correctly given is that the GoP does not recognize Afghans in urban settings as refugees. This exclusion appears to be attributed to donor and host country pressure.

55. The second issue, of how to deal with the present, recognized caseload, is based on the tacit, but unanimous acknowledgement that many Afghans will never return. UNHCR has been employing a strategy of progressive budget cuts: the health/nutrition budget was cut by 29% between 2003 and 200456, and by a similar amount between 2002 and 2003. Primary health care – like primary education – appears to be a pull-factor for camp-dwelling refugees to stay in Pakistan. The numerous Afghan urban slum dwellers who have not repatriated is evidence that the economic pull-factors are more important than the provision of services. The GoP clearly recognizes that, in the same way as in 1995, camp dwellers would rather move to the cities than “home”. ‘Decreasing basic services like health and education, and subsequently opting for closure of camps by relocating Afghan Refugees within Pakistan, is not a feasible option’.57 Therefore, it can be argued, that decreasing assistance, as a means of differentiating those who want to return and those who will stay does not work, at least not on a significant scale.

56. Admittedly, UNHCR is ‘often at the mercy of its donors and host governments’.58 That UNHCR cannot or does not want to provide and sustain services that are above host county level, for a population of “not-so- straightforward” refugees, is a justifiable attitude. The strategy of progressive budget cuts and the shifting of the financial burden to others has convinced neither the refugees nor the GoP. UNHCR needs to spell out its overall objective for Afghan refugees in Pakistan for the next 2-3 years. Spelling out such an objective is very tricky. On one hand, transmigration planning is not within the remit of the UNHCR, but on the other, UNHCR must at least plan for a transitional phase between post- emergency care and the eventual status of those refugees who have decided that they will not be returning to Afghanistan. In any event, these issues are way beyond the

55 WHO activities were not evaluated. During the briefing, it was stated that ‘WHO acts for refugees in non-camp situations’. While it is true that WHO’s activities do not target the camp refugees, no evidence was found that during this current post-'9/11' period WHO provides a significant complement to UNHCR’s activities, which would benefit the urban refugees. 56 From US$ 6,714,563 to US$ 4,824,855. 57 Agenda of Tripartite Commission (2003), op. cit. 58 Loescher cited in Turton D, Marsden P (2002), op.cit.

26 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 scope of this evaluation. Others have recognized the importance of the problem and of UNHCR’s assistance, and have appealed saying that ‘donor governments should not only increase their direct support to UNHCR’s protection activities… but they should also support these activities indirectly, by giving significant burden-sharing aid [to Pakistan]’.59

Conclusions

Without spelling out what UNHCR’s overall policy will be towards the Afghan refugees in Pakistan within the next few years, the current strategy of progressive budget cuts to the health programme does not give a convincing message for the future either to the refugees or the GoP.

Recommendations

Include urban refugees in UNHCR’s health policy framework. Base the UNHCR health policy on general UNHCR policy towards the Afghan refugees in Pakistan. The current strategy of progressive cuts should not be used ‘blindly’, in the absence of a clear policy, to deal with the future status of the refugees.

Lesson Learned

If UNHCR’s health policy is resource-, instead of needs-based and conforms to a definition of refugee status imposed by the host country, UNHCR risks missing out on half of its ‘clients’, i.e. the urban refugees, who are arguably the most vulnerable ones. Strategies of decreasing assistance as a means to differentiate those who want to return and those who will stay, does not appear to have its “intended” effect. Those hardest hit —i.e. the refugees —by service cuts are prone to move to cities in Pakistan rather than back to Afghanistan. The ones who were recently most likely to return —i.e. the urban poor — are not hit by cuts, as they do not currently benefit from these services.

5.3 Assistance Strategy

Implementing Partners

57. UNHCR works with about a dozen Implementing Partners (IPs) for its health programmes, including governmental, non-governmental (national and international) and quasi-governmental ones such as Pakistan Red Crescent Society (PRCS) (see Annex 4). Most IPs are involved in the delivery of PHC services, others have a disease-specific role or concentrate on patient referral. While the majority of NGOs receive co-funding from other donors, some NGOs receive no funds at all from UNHCR and therefore prefer not to be perceived as IPs. However, UNHCR still

59 Turton D, Marsden P (2002), op.cit.

27 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 insists on signing an MoU in order outline responsibilities and to achieve the necessary coherence.

58. Following UNHCR’s decision to discontinue working with PDH as its IP in Balochistan, new IPs were chosen through a well-planned selection process. Lessons can be learned from this process (although some IPs have asked for more transparency with regard to selection criteria, arguing that preference was given to the criterion of high co-funding, and saying that the selection was ‘conducted like an auction'. There was general agreement among IPs that, if UNHCR were to hand out all contracts through a periodic tendering process, there would be enough qualified IPs to go around. If the selection, and monitoring and evaluation processes were transparent, it would provide a performance guarantee.

59. UNHCR field officers gave the performance of the current NGO IPs good marks, often highlighting ‘decentralized management’ and ‘good community participation’. Lower marks were given for ‘poor management’, mostly to local or regional NGOs. By far the worst rating - confirmed by many other respondents - was given to the Project Directorate for Health (PDH). The PDH therefore merits special attention.

Project Directorate for Health

60. The PDH was created as a governmental organization more than 20 years ago, in order to help the GoP and UNHCR face the first refugee crisis. In the provinces, PDH is part of the Provincial Commissioner of Afghan Refugees (CAR). All Provincial CAR offices are linked to the office of the Chief Commissioner of Afghan Refugees of the Government of Pakistan (CCAR), which is in turn part of the Ministry of States and Frontier Region (SAFRON). Therefore, it is in no way related to the MoH, but parallel to it, as described by Rowland.60 Apart from some administrative positions in Islamabad funded by the GoP, the PDH structure and operations are entirely funded by UNHCR.

61. During field visits (including some ‘surprise’ visits) to BHUs, the evaluators found their performance roughly comparable to the performance of NGO BHUs. However, a number of shortcomings, mainly at the managerial level, were reported:61

1. PDH fulfils two, occasionally conflicting roles: a) coordinating NGOs and b) being an IP at the same level as, and often in competition with, the NGOs. There is little evidence that they are effective in their coordinating role. Instead, instances of ‘bullying’ through its government connections have been reported by NGOs.62 2. There is evidence that decision-making is over-centralized. Field Support Medical Officers (FSMOs) at the district level have no

60 Rowland M et al (2002) Afghani refugees and the temporal and special distribution of malaria in Pakistan. Social Science & Medicine 55 (2002) 2061-2072. 61 The following concerns mainly NWFP province, but similar reports were made in Balochistan, where cooperation with PDH as an IP has been discontinued. 62 The Ministry SAFRON, for instance, decides which NGOs are allowed to work in refugee camps; PDH is part of SAFRON. Another instance was the unilateral decision of CAR to stop user-fee collection (see below).

28 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

decision-making power; they ‘cannot even change a driver or a guard’. 3. A lack of peripheral transport has been reported, even though ‘most of the Deputy Directors in HQ office have three cars, with at least one for the family’.63 4. A high staff turnover in BHUs has been reported and observed. Staff members are transferred frequently and haphazardly, to the point of being in a ‘state of permanent rotation’.64 5. There is overstaffing, especially at the central provincial level:65 ‘PDH in Peshawar has six Deputy Directors, an army of EPI co- ordinators, Master Trainers, MCH supervisors and tens of support staff’.66 Following the budget cuts, PDH has evidently reduced services, but not staff. 6. Absenteeism at BHU level is frequently reported by UNHCR. 7. ‘Reluctance to deploy female doctors’.67 There is no financial incentive for Female Medical Officers (FMOs) to work in remote areas. 8. Corruption and financial irregularities are common. The recent embezzlement of the equivalent of US $300’000 by an administrative officer at PDH in Quetta led to their replacement in Balochistan with NGOs in 2004. The decision was supported by CAR. In August 2003, in NWFP province, an external audit found ‘most likely error’ (i.e. embezzlement) of the equivalent of US $ 360’000.68

Overall, it appears that PDH is much less efficient than NGO IPs.

Monitoring and Evaluation

63 Consolidation of the PDH Structure. Proposals for the year 2004. Confidential report by UNHCR Public Health Officer, 27 October 2003. 64 Assessment of performance and professional Integrity of the Project Directorate of Health (PDH), Dr. Ngogo, Health Advisor to UNHCR Peshawar office, June 2003. This is a very objective, frank and hard- hitting report that is concurrent with the findings of this evaluation in most respects. I found Dr. Ngogo’s assessment of PDH HQ in Peshawar to be correct, but somewhat hard on the health service (BHU) operatives, but then, perhaps we were shown the better BHUs. However, this report is sloppily put together and therefore does not have credibility in some quarters. Numerical rating of the various IPs generally concurrent with the findings of this evaluation.n 65 UNHCR sub-office Peshawar. Health Sector Profile and Plans for the Year 2004. Public Health Officer UNHCR NWFP. 18 September 2003. 66 Ibid. 67 This was convincingly documented by UNHCR in the report on the Consolidation of the PDH Structure (2003), op. cit. 68 The audit found that these irregularities consisted of overpayment, e.g.: drugs purchased at rates higher than market price’; ‘non-transparent tendering process resulting in irregular purchases’; ‘excess and unauthorized withdrawal of pay and DSA’; ‘procurement of medicines in excess of requirements’; ‘overpayment of staff’, etc., etc. According to oral information from UNHCR staff, this audit report has been recently “rectified” (i.e. altered): allegedly, there were errors due to missing documentation. It is also alleged that money changed hands prior to the ‘rectification’ of the report. This has made things worse, not better.

29 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

62. To ‘monitor and evaluate the effectiveness of the services and adjust as necessary’ is part of UNHCR’s standing orders. UNHCR field officers have a good grasp of the general (i.e. non-clinical) performance of IPs and their BHUs through their regular contact with the refugee population. However, with the exception of the two disease-specific IPs —HealthNet International (HNI) and the Association for Community Development (ACD)— who conduct regular supervision, formal monitoring and evaluation was generally found to be weak. This was recognized by UNHCR, “continual monitoring of health activities by the PDH senior staff as well as by UNHCR was very weak… in the case of UNHCR it was attributable to understaffing of the health unit”.69 Currently, UNHCR conducts no formal monitoring and evaluation of health programmes or BHUs. However, a programme that is not monitored is difficult to evaluate. The above-mentioned formalization of the ‘contracting-out’ process, if considered, should be complemented by the introduction of periodic monitoring and evaluation exercises according to criteria elaborated and agreed upon by UNHCR and its IPs.70 Such exercises would entail a re-enforcement of the UNHCR health management, which should be entirely justified by the size of the budget. The exercise should be participatory, (not only at the IP level, but also with the refugees), but it should not be conducted by PDH or any of the other main stakeholders.

63. Lastly, UNHCR’s monitoring capacity with regard to the drug procurement of PDH is weak and ineffective. In a letter sent to PDH in NWFP.71 UNHCR attests that ‘UNHCR Finance Officer has visited [your] office on 7 May 2003 to observe [the] drug procurement process… His mission findings were highly satisfactory as well’. Less than 3 months later, an external audit found that a ‘likely error’ of an amount of money equivalent to US $360’000 was, in large, part related to drug procurement.

Going Where?

64. The impact of progressive budget cuts made during the last two budget exercises has been felt at various levels. However, the main finding is that the objective has not been changed while the means have been significantly reduced: ‘UNHCR does not ask us to submit a new project, but to do the same with less money’. Some of the richer international NGOs have been able to compensate with bilateral funds, this is not possible for other NGOs. It appears, however, that UNHCR does not have comprehensive information about the co-funding of IPs.

65. Because of the cuts, some programmes have simply been cut, such as AMDA’s referral programme for chronic patients in Balochistan. Alternative funds were found, but only for another six months, for other programmes such as the one focused on reproductive health run by ARC in Mohammed Khel. Rationalizing the existing number of BHUs is common, but it still leaves a workload of preventative activities to be carried out in places where BHUs have been shut. Refugees were reluctant to seek care at a BHU in another camp because of differing ethnicities and

69 UNHCR Sub-office Peshawar. Health Sector Profile and Plans for the Year 2004. Public Health Officer UNHCR NWFP. 18 September 2003. 70 The list should at least include managerial capacity (efficiency), range of services provided, quality of preventative and clinical services, sustainability, and gender issues. Help may be sought in Afghanistan, where first experiences are made on a large scale with contracting out of health service provision 71 Letter of appreciation to PDH NWFP, by SO Peshawar. 03/Pesh/Progr/102.

30 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 large geographical distances, which represents a particular obstacle especially for women.72 In a number of documented instances in NWFP, BHUs cared for 25,000 refugees, while the UNHCR’s norm for BHUs is between 10,000 and 15,000 refugees (Annex 4).

66. Criticisms were also made by IPs with regard to the process for negotiating budget cuts, e.g. lack of transparency about the expected percentage of cuts. In most cases, cuts were negotiated with the IPs, but in some instances, IPs simply got their budget unilaterally cut, (moreover, without having time to change the project proposal), condemning them to under-perform from the start.

67. After two years of significant budget cuts and with probably more to come, it is suggested that UNHCR look more systematically at the present and future impact of further cuts, and discusses them with their donors. Further major budget cuts will most likely mean that there need to be changes in the objectives of the programme. This recommendation was recently made to UNHCR, by Macrae, who also pointed out the wider implications: ‘it will demand more robust means of analysing and reporting on the implications of funding shortfalls… at present, donors are unconvinced of the link. This suggests a move towards more need-based budgeting…’73

68. UNHCR wishes ‘to involve other partners’ and to broaden the funding base of its IPs, as a means to offsetting the decrease in its contribution. This raises the question of how sustainable its health programmes are. The care provided for by the MoH at secondary and tertiary level74 to the refugees, including the extremely low level of primary health care given to urban refugees is cause for concern. The provision of PHC in refugee villages is utterly unsustainable in its present form. This will not significantly change with an increase indirect funding to IPs. Despite the ‘growing pressure to adapt more developmental approaches to protracted crises’.75 UNHCR is applying an emergency strategy to a problem that has become perennial.

69. If UNHCR keeps cutting the budget, who will pick up the tab? Opinions about the capacities of IPs to fund programmes directly are divided. While at Headquarters, some say that there are no problems with cuts ‘as long as bilateral donors do not lose interest’ and point at the actual increase of bilateral funds; optimism in the field is limited: ‘the IPs are funded by the same donors who fund us.’ Unfortunately, the evaluators were not able to question the donors directly. Because UNHCR’s main IP, PDH is a government institution, it cannot broaden its funding base, making it even less sustainable than NGOs.76

72 Therefore, the suggestion was made to reduce the number of days of services, as also practised, rather than the number of BHUs. 73 Macrae J (2002) The bilateralisation of humanitarian response: implications for UNHCR. UNHCR EPAU/2002/15 December 2002. 74 UNHCR does provide a token assistance to tertiary hospitals that care for Afghan patients 75 Ibid. 76 Difficult therefore to understand the reasoning behind proposals such as Malaria and TB control activities currently undertaken by NGOs will gradually be incorporated within the PDH structure. This will require extensive training of the PDH staff in the year 2004 (UNHCR sub-office Peshawar. Health Sector Profile and Plans for the Year 2004, op. cit.).

31 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

70. Despite UNHCR’s own standing recommendation that ‘rapid integration with the MoH is essential,77 the MoH is currently not involved at all at the level of provision of PHC. However, previous attempts to involve the MoH were not encouraging; this is reflected in the example of a TB laboratory established by the Italian government in 1987 and handed over to Department of Health in 2000, the laboratory was unfortunately not sustained.

71. Efforts have been made to ensure that the health programme was more sustainable after UNHCR/WFP decreased their assistance level in 1995. In response to this, most health IPs started an intensive community mobilization and motivation process. As recommended by general UNHCR policy, user fees were introduced for curative services, while preventative services remained free. However, in 2002 due to a lack of clear procedural guidelines, CAR made a unilateral decision to simply stop collecting user fees. (CAR states that the reason was that the NGOs could not give a clear account of what they were doing with the user fees collected from refugees.) Currently all IPs charge a token fee of only 5 Rs. (less than US $0.10).

72. The imperative to find more sustainable solutions calls for more radical, instead of incremental change. Some NGOs have taken a new direction, e.g. Mercy Corps (MC) in Balochistan. MC funds five local IPs who provide PHC services to urban refugees and operate in close contact with the communities they serve. MC also provides capacity building in organizational development and a start-up supply of drugs. Curative consultations are charged at the equivalent of US$0.40-0.50, while drugs are sold at cost. As much as possible, MC and IPs tap into local sources like the MoH for free anti-TB drugs or heavily subsidized contraceptives or WHO, which has promised to provide anti-Leishmaniasis drugs.

73. In short, this model is situated between a private, for-profit model (which is the norm in Pakistan) and the virtually free humanitarian services offered by UNHCR. Refugees we visited, in a slum adjacent to one of the BHUs, complained about the fees charged, but continued to use its services (“because the fees being still at least less than 50% of market prices”). MC itself reported an overall increase in patient numbers during its second project year. Such BHUs are far from self-sufficient78, but, considering the higher degree of local ownership and increased efficiency, they have the potential to be more sustainable, while still providing the full range of preventative activities.

74. UNHCR's strategy of sticking to the programme objectives and assistance strategy while at the same time decreasing the budget will one day reach a breaking point after which most IPs will fail to perform. A new assistance strategy that would encourage increased sustainability would of course represent a medium-term investment. Interestingly, a proposal in the direction indicated has been made by CAR itself, though in an unusual place,79 where the following objective is mentioned, ‘after demobilization of GoP / UNHCR Medical Units, coordination and setting up

77 UNHCR (2000) Handbook for Emergencies. Second Edition, UNHCR Geneva. 78 The four MC-funded BHUs that have been functioning for a whole year between September 2001 to August 2002 report a mean yearly income from fees and medicines of US $ 5’500. Their total cost is not available, but as a comparison: the average yearly cost of a BHU of FPHC is US$ 23,600 (Frontier Primary Health Care, Annual Report 2002). 79 Role and charter of duties – medical coordinator UNHCR/CCAR, Islamabad (based in warehouse Islamabad)

32 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 of alternative Health Care Units on self-help basis or introduction of small payments for providing curative treatment to Afghan refugees.’ Such ideas are not completely new to UNHCR. In Quetta, they have stated their intention to start a pilot with an INGO in order to build local capacities.

Conclusions

IP performance, according to UNHCR’s own rating and the evaluator’s observations, is generally good, though there are a few opportunities for improvement. By far the biggest shortcomings appear to be the insufficient performance (especially at managerial level) of the PDH. In the absence of a real ‘exit strategy’, UNHCR is applying an emergency strategy to a problem that has become perennial. The provision of externally funded, primary care services, far above the local level, for a population that largely will not be repatriated, is not a sustainable solution. There is no formal monitoring and evaluation of the general performance of IPs. Specifically, UNHCR’s ability to monitor the drug procurement of PDH is weak and ineffective. UNHCR’s successive budget cuts are starting to jeopardize the standards and quality of care being given by IPs who were not able to offset them through other funding sources.

Recommendations

Improve the current unsatisfactory performance of PDH. This can be done in three ways: Follow the example in Balochistan, and discontinue funding PDH as an implementing partner. A small number of senior qualified staff would fulfill the liaison function with the MoH/GoP. In order to maintain valuable field experience, PDH clinical and peripheral management staff could be encouraged to form an NGO. To restructure PDH. Currently, their conflicting roles, of coordinating and implementing services, needs to be separated. PDH’s provision of PHC needs to be subject to the same evaluation criteria as any other NGOs. The project’s administration needs to be downsized to the level of the CAR education cell. An external management consultant should lead such a restructuring exercise, with involvement of UNHCR and CAR. To improve the status quo as a ‘minimal option’. This should include management decentralization the handing over of the drug purchasing to the UNHCR, improved staff management and, most of all, closer UNHCR supervision and regular monitoring and evaluation. Introduce a formal performance monitoring mechanism and periodic evaluations according to the criteria elaborated on by and agreed upon with the IPs. At the very least, these should include managerial capacity (efficiency), range of services provided, quality of preventative and clinical services, sustainability and gender issues. The refugees as service users must be included in these exercises. This task will not materialize without additional expense. However, this expense should

33 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

be seen as a cost saving measure as so many funds are currently lost due to a lack of monitoring and supervision. Introduce periodic, formal and transparent tendering processes for contracting out services to IPs, in connection with the above-mentioned monitoring and evaluation process. This should prevent standards from slipping and provide a guarantee to UNHCR of value for money. The additional administrative burden that this represents needs to be taken into account. Start a collective and transparent consultative process in Islamabad with donors and IPs regarding the funding situation between IPs and UNHCR, instead of simply imposing a budget without discussing their implications for implementing partners. Plan strategies that increase the sustainability of health care. In view of the possible prolongation of the refugee problem, new objectives must be formulated on a strategic level. Cost-recovery should be re-introduced, but in a concerted and professionally administered way, potentially with the help of an external consultant. The potential for local NGOs to provide care in a more efficient way — by involving the communities — must be fully explored and implemented, where possible, by managerial capacity building. Include the MoH in policy considerations and as a partner in the planning process.

Lessons Learned

UNHCR will not be able to keep on cutting budgets, while expecting that health objectives will remain unchanged. There may be a middle ground between providing unsustainable, comprehensive, nearly-free humanitarian PHC services for camp refugees and the sub-standard level of free public care and variable ‘for-profit care’ available to poor Pakistanis and urban refugees. Local NGOs originating from, or rooted in, the refugee communities may be engaged to work in a more sustainable way by providing subsidized services below the private market price, which significantly contribute to the provision of health care services. A special government structure for refugee assistance created and funded by UNHCR risks growing up to be another sorcerer’s apprentice, if it is not constantly and tightly monitored and controlled.

5.4 Health Information System

“… the data provided on a monthly basis is completely irrelevant and does not help at all in making health management decisions, or reorientation of our programmes….: it does not provide a comparative summary or any analysis”.80

75. Afghan refugee camp populations are quite fluid. Most camps keep track of their populations by a combination of community health worker (CHW) reports and UNHCR field officer checks. However, there is considerable difficulty in getting the population figures correct because of refugee movement. Particularly in the old

80 UNHCR sub-office Peshawar. Health Sector Profile and Plans for the Year 2004, op. cit.

34 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 camps, some adult men are visiting Afghanistan as frequently as twice a month and many of these men leave the camp regularly to find work. Sometimes they take relatives with them, and sometimes they bring relatives back with them. It is quite common for a refugee family to have family members in a camp, family members back in Afghanistan, as well as other family members living in urban areas in Pakistan.

Data Collection and Processing set-up

76. Data is collected from CHWs, FCHWs, CHW supervisors, BHU registers, record cards and monthly reports. The main denominator data (i.e. births and deaths information) is collected by the volunteer Afghan refugee CHWs. Male CHWs may have minimal education and females usually have no education other than the basic training provided by PDH and supervisory NGOs. The training curriculum shown to me by the Deputy Director for Primary Health Care at the Peshawar PDH office does not include any training on HIS data collection. It appears that no verification of data at source is made either on a routine or random basis. Collection of denominator data seems therefore to be the weak link (basis) to the whole system of this HIS.

77. Data on malaria and Leishmaniasis are collected by Health Net International (HNI) from passive case detection at 115 BHUs and 62 field laboratories, and sent directly to the IRC HIS coordinating unit which adds them directly into the HIS country report.

78. Data on tuberculosis are collected by the Association for Community Development (ACD). ACD supports 100 detection and treatment clinics and 47 field laboratories. The data are sent from BHUs to the provincial HIS offices. No data appear in the HIS report to October 2003. The figures are estimated to be about 3800 new cases detected per annum; TB mortality is about 2%.

79. Data are collected mostly by people with little formal education (and in some cases illiterates e.g. the female community health workers), and compiled by health workers with only ‘workshop’ training in data collection and management. However, there are up to eight complex tally sheets, five stock forms, and 15 demographic and mortality/morbidity registers to be filled out on almost a daily basis. In addition, there are 17 complex monthly reporting forms and at least 3 quarterly reporting forms to complete which may explain why they are poorly filled out and that the central unit is given incomplete data. The data collection and BHU level compilation of data needs to be simplified. It is equally important that the IP’s ability to analyse data be strengthened considerably.

80. Data are collated at the BHU, and then sent to the Provincial HIS director who combines data from the entire province. Although there seems to be reasonable collaboration and between the provincial and IRC Coordination Unit, the Unit has little influence on the information gathering process, including the way data are collated and reported from the Provincial HIS and the individual BHUs.

81. The IRC coordination unit then combines the HIS data from the three provinces (Balochistan, Punjab, NWFP) and makes a ‘country report’. The national coordinating unit is currently using an Excel software system, which was set up by

35 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

CDC. They have experienced a number of problems with the software since they started using it over 5 years ago.

82. There is new software in the pipeline. In fact, the Director of the HIS country coordination unit, Dr. Ishaq, recently returned from a workshop organized by CDC in Bangkok that was specifically organized to train staff on the new Access-based software package. Dr. Ishaq said that they would not introduce the new software until the new funding contract is in place. At the time of writing, some health units were recording data on both the new Access-based software and the Excel spreadsheets.

Reports and Feedback to UNHCR and NGOs.

83. Dr. Naveeda Rehman, UNHCR Health Project Officer in Islamabad reported that she had not received any HIS Coordination Unit reports since receiving the July- September 2003 quarterly report in November 2003 and had only received separate provincial HIS reports.81 Most NGOs state that they almost did not receive feedback on the information they sent in or copies of the coordination report. Additional information was sought from IRC, SCF, WAMY, ARC, PDH, and BHUs. The comments below are based on the reports from the HIS coordination unit, viewed by the evaluator. The reports consisted of the July-Sept 2003 quarterly country reports and a cumulative country report as of October 2003.

Comments on the HIS reports

84. General Comments. The reports are well presented. They are spiral bound and colour printed. The data is presented was virtually without analysis or comment. The denominators and methodology are not explained. There are many anomalies and peculiarities in the data (see below).

85. Population Data: Male female ratio is approx. 1:1 and the total population is 1,631,287 million.

86. Births recorded are only 4,471 giving a CBR of 2.7/100; this seems to be a gross underestimation because:

PDH has been assuming a CBR for over 20 years to be 4+; The total fertility rate of Afghan refugees (by all reported surveys) has shown it be 6.8 or more; The figures for the March 2003 census of refugees show that the children < 1 year were enumerated to be 62,300 which is considerably more than the annualized total birth figures for the HIS; The raw figures for the Union Aid NGO field area in Peshawar, for the nine camps under their care (Total population of 185,863), reveals a CBR of four+ for every camp measured.

81 At the feedback session in Islamabad on 17.12.2003, it was claimed that the reason that his unit had not produced any reports for 5 years was because they had been developing the software.

36 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

87. Underestimating the births gives a falsely low figure for all birth denominated indices, e.g. stillbirth rate, NND rate, PMR; it also falsely improves the figures for immunization coverage, etc. Based on the collaborating data from the various sources, the actual birth rate may be at least 20% higher than the levels currently being reported.

88. Mortality data is even more grossly underestimated. For the 10 months prior to 31 October 2003 there were only 351 deaths recorded by the IRC HIS coordinating unit. The fact that this is a gross underestimation of the number of actual deaths is suggested by:

Only 267 deaths recorded in the 185,863 population of the nine camps cared for by the Union Aid NGO; compared to only 265 recorded for the whole of NWFP by the IRC HIS coordinating unit; A CDR of 0.22 recorded in the HIS reports is so grossly low that it should have alerted the unit to data errors. Moreover, it is not clear from the HIS reports whether this figure (and indeed many other rates quoted in the reports) are monthly figures, or cumulative figures – the figures do not appear to be annualized; The most anomalous figures are reported for female mortality which is about 30- 40% less than those reported for males in the HIS reports, to spite the fact that the morbidity data shows that more females than males present to BHUs with illness; and Maternal mortality, infant mortality, neonatal mortality, stillbirth mortality rates are all reported at such low rates that the unit should have suspected that there were omissions in the reports 89. Morbidity Data. It is not clear what the morbidity data refer to - as is the case with much of the data. The number of cases of morbidities recorded in the October report (claims to give cumulative figures) is less than that reported in the quarterly July- September report; e.g. URTI 51,231 cumulative to 31.10.2003, but 128,669 for the quarter, July-September 2003.

90. Reproductive Health Data. Only one maternal death is reported for Balochistan and none for the other provinces in the October report whereas eight were reported in the quarterly report July-Sept 2003. Moreover, anecdotal data from the various camps visited on this mission indicate that many maternal deaths are not reported to the BHUs as maternal deaths. The underreporting of births gives a falsely high level of ANC and postnatal care coverage as well as an inflated estimate of the number of supervised birth.

91. Contraceptive prevalence data is variously reported as 1.14% (July-Sept quarterly report), 3.36% (October 2003 report); 5-9% by various NGO implementing partners. However, it is not clear from the HIS reports just how their CPR is calculated.

92. EPI Data. The report indicates gross wastage of vaccines (up to 50% in some cases); but it is not clear how this is calculated, or whether it is an accurate representation of the true level of waste. Reported coverage is based on underestimates of children due to the underreporting of births.

37 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

93. Referral Reports. Only 1,164 referrals were made to outside camps facilities, but this does not include the many self-referrals by the refugees themselves

94. Malaria, Leishmaniasis and TB Data. The malaria and Leishmaniasis data are taken directly from the HNI database. No TB information is provided in the October report, although the figures for TB for the July-Sept quarter 2003 are pasted in from ACD and BHU data.

Conclusions

After five years of operations, the national HIS coordinating unit system has yet to provide any reliable or useful compilation or analysis of data for the BHUs, NGOs or UNHCR. Overall, the HIS Coordinating Unit reports do not demonstrate either the ability to accurately collect data or a clear understanding of data analysis. The Unit needs to establish a data collection system based on accurate data collection and reliable information from the district/camp/BHU/responsible agency level as well as at the provincial and national levels. Most IPs (at the BHUs level) have a reasonable system for collecting data; however, all rely upon poorly-educated (and in the case of voluntary female community health workers, usually illiterate) community health workers to collect denominator data. Nevertheless, it was pleasing to see how well most of these CHWs were able to collect data using their network of refugee assistants and contacts. The system in Haripur where FCHWs have books of pictorial stickers that they paste onto their reporting forms works very well. The PDH BHUs are not nearly as well organized or well set up for basic data collection as most of the NGOs. A number of IPs did admit however, that mortality data was difficult to collect (especially stillborns, neonatal deaths and female child deaths). Some IPs have even tried to validate their mortality data collection by carrying out daily checks of the cemeteries. A number of independent surveys and reviews were carried out over the past several years82,83,84 and a census was conducted in March 2003. These sources provide very accurate figures on the refugee population and its structure. Additionally, a number of NGOs have produced annual reports with good summary data85,86,87,88.

82 Bartlett et al. Maternal Mortality among Afghan refugees in Pakistan, 1999-2000, Lancet, 23.2.2003, Vol. 359 643-649. A most useful independent study on the area in Hangu that was visited during the evaluation. 83 Tomeczk B. et al. (2000), Finds from a Reproductive Health Survey of Afghan Refugees in Pakistan. A collaboration between CDC and IRC. 84 Women's Commission for Refugee Women and Children (2003), Still in need: Reproductive Health Care for Afghan refugees in Pakistan. 85 IRC Primary Health Care Programme in Hangu District, December 2003. 86 Save the Children Fund (US), Participatory Rapid Appraisal, “A study on sustainable primary health care services” Afghan Refugee camps, Haripur and Ghazi, 8-27 July 2003, Shaima Suleman Kheil et al. A good survey of SCF work in Haripur. 87 Quarterly report referral management system in Quetta and Charman July –Sept 2003, Association of Medical Doctors of Asia. 88 Frontier Primary Health Care, Annual Report 2002.

38 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Since its inception five years ago, the national HIS coordinating unit has produced only two country reports; a quarterly report for July, August, September 2003 and a cumulative monthly summary report at the end of October for the year 2003. Some of the figures in these reports are clearly wrong, but there has been no attempt to analyse any of the figures. IPs and the UNHCR confirm that they have received no useful feedback of figures or statistics over the past 5 years since the national coordinating unit has been in existence.

Recommendations

A rectification process needs to begin immediately, (this should be done before any additional funds are spent on new software, training or any further support of the HIS coordinating unit by IRC/CDC), this process could proceed by: Project evaluation of the HIS coordinating unit carried out by a community health specialist with biostatistics expertise; or A tripartite review of the collaborative HIS system by the three interested parties in Islamabad (CDC, IRC and UNHCR, with representatives of the users/IPs), and facilitated by a community health specialist with expertise and experience in HIS in developing countries. The local HIS loop of data collection, compilation and local feedback needs to be strengthened. It is working quite well for most NGOs, but not so well for others (it is not really working at all in PDH units). It needs to be standardized (albeit with a modular approach that would allow some IPs with more services and a desire to analyse them to do so). Some NGOs will require additional assistance in analysing the data, in order to detect the early warning signs of problems, and to help them fine-tune their services, etc. The coordinating unit needs to have expertise in collating figures from the provinces (which seems to be their main function at this moment), verifying the authenticity of the data collected, and in providing analysis and feedback of the data. This should not only go upwards to UNHCR and the Pakistan government, but also to their implementing partners and clients — top-down HIS planning and reporting systems rarely work.

Lessons Learned

Most IPs have an interest in their work to the extent that they collect useful data and analyse it for their own purposes. A top-down approach —no matter how sophisticated the software89 — is unlikely to produce useful results and analysis unless it is supported by data collection that can be validated. Analysis of HIS data requires more than expertise in using data entry software.

89 This in contradiction to UNHCR’s own guidelines that the HIS should be kept simple (source: UNHCR, 2000, op. cit.)

39 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

5.5 Coordination

95. To provide an integrated and effective service to refugees there needs to be good coordination between various operational partners (OPs) and implementing partners (IPs). Coordination should happen at different levels, between sectors (such as water and sanitation), with IPs and OP’s and with government bodies such as the Pakistan MoH.

96. Officially, PDH is supposed to take the lead role of coordination, since it is the biggest health IP and also an OP. In reality there is conflict of interest with PDH carrying the roles of IP and OP. since it is the largest IP, its role as coordinator gives it an unfair advantage. This needs to be addressed.

97. PDH was set up as the GoP’s health implementing wing. It should have strong ties with the MoH, but, in fact, it works as a parallel structure. The PDH and MoH have informal contacts but no formal regular coordinating meetings. As far as it is known, there are no regular coordinating meetings between PDH, other IPs and UNHCR in NWFP. This makes coordination haphazard and unsatisfactory. Furthermore, the UNHCR technical post is vacant which also affects coordination. Some of the IPs and OPs do a good job of trying to co-ordinate their health and environmental services within the refugee camps, while others are not doing a good job. For example, WESS is an IP, which is mainly concerned with provision of water and sanitation to the camps in Balochistan province.90 They need to better coordinate their activities with the health care providers. Some OPs are not really involved, e.g. UNFPA, which is a great pity because there is a big health care gap for IPs in the area of reproductive health. WHO, UNICEF and WFP provide integrated assistance in their own specialized areas. UNHCR should assume the role of coordinating these activities and provide input to the various IPs and OPs.

98. In Balochistan, there is a good system of fortnightly meetings, hosted by the Quetta UNHCR office, where the various OPs and IPs meet to work out better, more appropriate coordination of activities. Perhaps the reason this has been more successful is that the programme is smaller with less IPs. However, UNHCR’s presence at these meetings is vital for decision-making, but the meetings have not taken place in the last two months (it is unclear why this is, maybe due to the lack of UNHCR health focal person). Peshawar and Islamabad offices could follow this model.

99. Although we did not do a comprehensive evaluation of all the IPs who are involved in provision of services to the camps, we did notice quite a variation in their attitude and effectiveness in coordinating services. Coordination between the different provinces, the Islamabad head office and the sub-offices is lacking. This leads to duplication and “re-inventing of the wheel”. Generally, there needs to be more coordination with regard to the development of standard protocols of clinical practice and training.

Conclusion

90 Numerical rating and comments on IP performance by UNHCR camp field officers and field clerks for Balochistan and NWFP.

40 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Shortcomings with regard to coordination — between UNHCR offices, between UNHCR and its OPs/IPs and between IPs — can lead, at times, to service overlap or gaps.

Recommendations

Introduce and conduct regular structured coordination meetings. Coordinate and standardise protocols for all IPs, in particular protocols around clinical practice and monitoring of programme activities.

5.6 Norms and Standards

100. There are norms and standards available for most aspects of refugee health care, particularly in the RH, child health and EPI areas. The text “Reproductive Health Care during Conflict and Displacement”91 produced by WHO, “Reproductive Health in Refugee Situations, an Interagency Field Manual”92 produced jointly by WHO, UNFPA and UNHCR are good documents. The former is for health managers and policy makers, and the latter for health workers. However, the management protocols in the RH field manual93 are too generic to be useful to most of the health workers in refugee camps in Pakistan: it would be more helpful to give specific guidelines or protocols of standard management to health workers for their day-to- day interaction with clients and patients. There is a gap here.

101. There are standard protocols and guidelines of patient management in the Mercy Corps documents and most BHUs have wall charts on the management of tuberculosis and diarrhoeal disease. In some BHUs, we found that there are few standard protocols available, sometimes there were loose pieces of A4 paper in drawers here and there, and sometimes they were in a file in the doctor’s office. There is a great need to have all standard managements and treatment protocols in one ready reference guide (a pocket manual of standard managements and protocols).

Conclusion

There is a dearth of standard protocols for the management of minor but commonly presenting ailments.

Recommendations

Each BHU should have a small reference library of appropriate clinical materials for the health workers to use. IRC in the Hangu area have quite good small libraries in their BHUs,94 but they lack clinical texts. Some general reference to clinical texts should be included in the BHU libraries e.g. ‘Primary Mother care and Population’95 and ‘Primary Childcare’.96

91 Reproductive Health during Conflict and Displacement, a Guide for Programme Managers, RHR, WHO, 2000. 92 Reproductive Health in Refugee Situations, an Interagency Field Manual, UNFPA, WHO and UNHCR, 2000. 93 Ibid. 94 List of library books in the IRC BHU libraries in NWFP, per Dr. Tila, Medical Director.

41 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

UNHCR should assist an NGO in drawing up a pocket-sized standard management manual of common illnesses found in Afghan refugee camps along the lines of Pocket Manual of Standard Treatments of Common illnesses in Children97 and the Pocket Manual of Standard Treatment of Common Problems in Obstetrics and Gynaecology.98 This guide should be regularly updated, widely distributed, and made available to all health workers in the system.

Lesson Learned

Generic manuals, no matter how carefully prepared by the international community, cannot be adapted for every day use by most health workers in developing countries.

5.7 Nutrition

102. Support to Afghan refugees in Pakistan in the form of a general food basket has been in place for many years. However, a strategic decision was made to phase out this assistance at the end of 1995 as part of a repatriation process. At that stage, some of the refugees returned to Afghanistan. However, many refugees remained in the old camps while others moved to urban areas. At the end of 2001, there was a new influx of refugees and food aid was restarted to this new group.

103. As numbers continually fluctuate, it is extremely difficult to have a true figure of the number of refugees but there is presently an estimate 1.2 million in camps and of this 215,000 are “new refugees”, receiving food aid assistance. There is a further ‘invisible’ refugee load in urban areas not receiving food aid either.

104. WFP is the OP responsible for the procurement, fortification and transportation of the food aid. IPs such as Shelter Now International, Inter SOS and IRC are involved in the end stage distributing it to beneficiaries. The main staple wheat, pulses and salt are procured locally, while oil is imported. In a new pilot study WFP is working directly with IPs in the transportation of food aid to the end distribution point (normally UNHCR is responsible for this), and in NWFP WFP has its own trucks for this distribution.

105. Monitoring nutrition status takes many forms. This includes ad hoc nutrition surveys, growth monitoring, annual food security assessments such as the Joint Food Needs Assessment Mission (JFAM) conducted by UNHCR and WFP99 and one-off studies such as the household food assessments in individual camps.100 WFP also collects monthly data on the terms of trade, such as market prices. In the BHUs, anthropometric data is collected on individual children.

95 ‘Primary Mothercare and Population’, King MH, Mola GDL, Thornton J, Bullough C, Guillebaud J, Spiegel press UK, 2003. (Available from [email protected]) 96 Primary Childcare, King MH, Savage, F, Bullough C, WHO publication, 2002. 97 Pocket Manual of Standard Treatments of Common Illnesses in Children for Doctors and Nurses in Papua New Guinea, PNG government printer, 8th edition, 2003. 98 Pocket Manual of Standard Treatment of Common Problems in Obstetrics & Gynaecology for Doctors and Nurses in Papua New Guinea, 4th edition, 2000. 99 WFP/UNHCR joint food needs assessment mission to Pakistan, 28.9.2003 – 10.10.2003. 100 Household Food Economy Assessment, Refugee Camp Shamshatoo, NWFP, by Sustainable development policy Institute (SDPI), to the World Food Programme & UNHCR, 2003.

42 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

106. Specialized feeding interventions, consisting of dry take home supplementary feeding programmes (SFP), restarted in the new camps in 2001, targeting malnourished children aged under five-years, pregnant and lactating women and the chronically ill. The SFP was stopped in September 2002, due to a break in food pipeline and when restarted in April 2003 a reduced ration was given (reduced from 1165kcal to 740kcal). No treatment for severely malnourished children is available at BHU level; instead, they are transferred to a referral hospital. In the old camps, children identified with malnutrition receive health and nutrition education but no food.

107. The Lady Health Visitors (LHVs) conduct routine nutrition education in the BHUs. This takes the form of group information sharing and occasionally cooking demonstrations. Breast feeding and complementary feeding information is given to new mothers.

108. Micronutrient deficiencies are always a concern in refugee populations. In both Afghanistan and Pakistan iodine deficiency, vitamin A deficiency and anaemia are major issues.101 In Afghanistan, scurvy outbreaks were reported in 2001.102

Food Security

109. There is a substantial physical difference between the old and new camps. The old camps had evolved into mud hut villages, similar to the structures they lived in at home. In Haripur and Thall camps, each household visited was enclosed by a wall for privacy and security. Normally brothers, their spouses and children live within this compound with the brother’s mother.

110. The economic situation appeared to depend on how long the family had been in the camp, and where the camp was in relation to local towns and economic employment prospects. Two of the households visited in the old camps had a family member working overseas (UK or Saudi Arabia). In the well-established camps, families had household goods such as furniture and carpets. Furthermore, some households had livestock and fowl within their compounds. In these well-established camps, each compound had its own latrine. In Haripur, each compound visited had its own piped water supply. Local work included daily labour, brick making, trading and driving.

111. However, this was not the case in the new camps visited. These are in general temporary, much more isolated, situated in an arid landscape and closer to the border. Many of the camps were still tented, those visited during this evaluation included, Latifibad, Dara 1 and 2, and Landi Karaci in Balochistan. In NWFP Asgaro camp was also a tented camp. The people in the tented camps appeared to be much poorer. The environment appeared more hostile; no livestock was visible, there were fewer employment opportunities and they were further from markets. In the new camps, more men were present suggesting that there were less work opportunities. In some of the camps water was an issue, being trucked to the camps daily.

101 National nutritional survey 2001-2002, Planning Commission Government of Pakistan, UNICEF, Pakistan. 102 Assefa F (2001), Scurvy outbreak and erosion of livelihoods masked by low wasting levels in drought affected Northern Afghanistan, , Field Exchange, August 2001, Issue 13, www.ennonline.net-

43 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

One woman in Asgaro camp had a few flowers growing in her compound. Her husband had been a farmer in Afghanistan. When I asked her if she intended to plant a vegetable garden, she said that water was an issue. She was dependent on her children to fetch the water, as she was not allowed to leave the compound. Her children were small.

112. The general food ration is just over 2100kcal/per person, with the appropriate percentage of calories in the form of protein and fats. The nutrition surveys that have been conducted over the past three years show acceptable levels of acute malnutrition, surveys in Balochistan 2001,103 in NWFP 2002104 and in Balochistan & NWFP 2003.105,106 The rates of global acute malnutrition (GAM) have remained less than 10%. Although there have been slight changes in the rates of GAM over the years the differences are not statistically significant. The GAM rate in Afghanistan is significantly higher at 25%107 and in Pakistan at 13.1%.108

113. However, this is not the case with chronic malnutrition or stunting. In the nutrition surveys conducted in 2003, there is a statistically significant difference between the provinces with much higher rates in Balochistan than NWFP. The rate for NWFP was 45.8% (CI 41.2% -50.4%) while chronic malnutrition in Balochistan in Chaman camps was 69.7% (CI 63.5% - 72.2%) and Mohamed Khel camps 58.7% (CI 53.8% - 63%). Furthermore, there is a statistically significant higher level of chronic malnutrition in Chaman camps. In both Afghanistan and Pakistan, the rates of chronic malnutrition are lower at 52% and 36.8% respectively.109,110

114. It is difficult to understand why there is such a difference in chronic malnutrition between Mohamed Khel and Chaman camps. Some of the reasons may include:

More transient camps in Chaman since it is closer to the border; Possibly a higher percentage of new refugees in Chaman camp; Water tankering in Chaman camps created issues around public health, such as hygiene and amount of water available may be less; More families living in tents in Chaman; Less economic opportunities in Chaman, as farther from Quetta and other semi urban areas; and General ration to camps in Chaman not as regular and not all the commodities are available.

103 Report of Nutrition Survey in Existing Afghan Refugee Camps in Balochistan, Pakistan December 2001, PDH, UNICEF, UNHCR, MCI. 104 Nutrition Surveys of Afghan Refugees Living in Old Camps and Urban Host Communities in NWFP of Pakistan, April/May 2002, UNICEF, UNHCR and PD. 105 Action Contre La Faim, Nutritional anthropometric survey, children between 6 and 59 months, new camps of Afghan refugees in Balochistan province, Pakistan; March/April 2003. 106 Action Contre La Faim, Nutritional anthropometric survey, children between 6 and 59 months, new and old camps of Afghan refugees in NWFP, Pakistan, 2003. 107 UNICEF (2001), State of the World’s Children. 108 Ibid. 109 National nutritional survey 2001-2002, Planning Commission Government of Pakistan, UNICEF, Pakistan. 110 Ibid.

44 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

115. In NWFP, many refugees were asked about the general food ration and there were no complaints about it. In fact, many were able to state the date of the month when they regularly received food, which suggests that the distributions are working well and are very regular. This is not the case in Balochistan where since September the ration has been on a five-week rotation, and the staple was reduced from 15kg to 9kg in November. This is highly unsatisfactory given that the temperatures during our visit were minus 10° Celsius. Therefore UNHCR should increase the ration by 400-600kcal/day111,112 rather than reducing it. Furthermore, these refugees are living in very poor conditions and many of the children we saw were without shoes, socks or coats.

116. This suggests that it is essential that the general food ration be continued in the new camps, since food security is much more fragile. It is also extremely important that this ration is regular and all commodities are available. The substantially lower temperatures in the winter in Balochistan Province need to be taken into consideration. It is likewise important to identify the reasons why chronic malnutrition is much higher in Balochistan, particularly in Chaman camps, and address them, and important to introduce blended foods as part of the general ration since it is possible that complementary feeding practices are very poor. Additionally, small children require more calorie-dense foods than adults and adolescents.

Growth Monitoring and Nutrition Surveys

117. It is extremely important to monitor the nutrition status in the under-five age group since this group is most vulnerable and particularly sensitive to change. There is an attempt at growth monitoring but it is carried out in an ad hoc manner and senseless. In order to monitor a child’s nutrition status, it is essential to regularly, monitor weight gain. In the BHUs, the focus is only on ‘at risk children’ that includes children with low birth weights (LBW), twins and premature babies. These infants are monitored monthly, and the others on a quarterly basis. At present, this monitoring is only done until the child is one year old, which should be extended to at least 18 months, preferably to thirty-six months.

118. Most of the BHUs visited used a register-book to record the children's weights and calculate the weight gains, static weights, or weight losses. This is not very useful, as it does not give any real indication of the individual child’s progress over time (a 100g increase of weight in a 3-month period is not adequate yet it would be registered as a weight gain). One IP staff member was using the 'Road to Health' card for recording this information on individual children but did not understand how to fill-in the card and filled it in incorrectly. The IP staff member said that this was the way it was shown to them by UNHCR.

119. The Balochistan HIS report for September 2003113 indicated that less than 1,000 children under five were measured in that month. The estimated population of children aged under-five was 78,753 which means less than 2% of the under-five population had any growth monitoring in that month. Of this group, only around

111 UNICEF 2001, op. cit. 112 Food and Nutrition Needs in Emergencies, guidelines jointly developed by UNHCR, UNICEF, WFP, WHO. 113 Balochistan provincial analysis of HIS report September 2003.

45 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

40% had gained weight. The HIS is very unreliable, lacking any analytical data. Even if the data were collected it would not be used well (see HIS chapter). There is even less analysis in the quarterly country HIS report.114

120. Ad hoc nutrition surveys have been conducted over the years. There were no surveys conducted between April/May 1997 and December 2001. Following the influx of refugees, surveys were conducted on a more regular basis. There was one conducted in 2002 and one in 2003. Presently these surveys are the only consistent form of data collection on the nutrition status of the refugee population, as the growth monitoring system is not functioning and the HIS is unreliable. However, the recommendations from these surveys have not been addressed, as growth monitoring has been suggested in the past.

121. There is confusion between chronic and acute malnutrition, and there is an extreme focus on acute malnutrition to the detriment of chronic stunting. In Chaman, less than five percent are acutely malnourished while almost 70% are chronically malnourished. This needs to be addressed. Chronic malnutrition suggests a long- term problem of poor and inadequate nutrition that needs to be addressed at the household level, with education and possibly resources. It is easy to recommend the correct food for young children, but the level of resources available needs to be considered. There is a need to understand why the rates are so much higher in Balochistan than NWFP. In the month of November, in Haripur BHU, high rates of low birth weights were not recorded. For example, out of the 72 recorded birth weights only three were low birth weights. This figure was similar in other areas, indicating that the nutritional status deteriorates after birth rather than intra-uterine.

122. Breastfeeding is the norm, at least in infants under four-months of age. Mothers, when asked about complementary foods listed different types of food. In general, 4-6 months seemed to be the time when the feeding started. It was not possible to get detailed and accurate information on real complementary feeding practices during this evaluation. Mothers continued to breast-feed for up to two years.

Supplementary Feeding

123. Supplementary feeding was restarted in April 2003 after a seven-month lapse. It is difficult to understand why it was restarted given that the rates of Global Acute Malnutrition (GAM) were extremely low at less than five percent. Moreover, the ration size was reduced by nearly 40% of the recommended ration in April 2003. This has meant that recovery from moderate malnutrition is slower. In the BHU in Dara 1, there were 34 children. Most of them had been in the programme for 2-3 months and had not recovered. Additionally, some defaulted for 1-2 weeks and then returned to the programme. Given that the SFP is in place it should be continued over the winter, particularly as the food is available but the ration size must be increased.

Micronutrient Deficiencies

114 National Quarterly HIS report (July-Sept, 2003).

46 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

124. There have been no reports of outbreaks of micronutrient deficiencies. The HIS does not record rates of anaemia diagnosed although some NGOs collect this data on anaemia under “chronic diseases”. IRC had a rate of 0.3% of those seen between January and October diagnosed with anaemia. There is no recent country data on micronutrient deficiencies for Afghanistan, but there were outbreaks of scurvy in 2001115 and iodine deficiencies are a major problem in some areas. In Pakistan during the 2001- 2002 nutrition surveys identified 36.5% of women as being severely iodine deficient with another 20% moderately deficient (urinary iodine excretion).116 Anaemia and zinc deficiencies are also high within the Pakistan host population.117 In Pakistan's host population, Vitamin A deficiency has dropped from 30% to around 12%. This is attributed to the many polio campaigns, during which they Vitamin A capsules are also distributed. The 2003 nutrition surveys118,119 in the new camps in NWFP and Balochistan indicated rates of 1.5% and 2% respectively for Vitamin A deficiency and 54.5% and 13.5% respectively for anaemia. The anaemia results are very subjective as the way of identifying the anaemia was by looking at the colour of the conjunctiva.

125. Signs and symptoms for micronutrient deficiencies are difficult to identify unless staff is well trained. Given the high burden on health staff to see numerous patients on a daily basis, it is likely that signs and symptoms of deficiencies are missed. Some deficiencies need invasive and expensive blood tests to confirm diagnosis.

126. However, it is possible that this may be a major factor contributing to the high levels of chronic malnutrition in Balochistan new camps, given that the food basket is poor, blended food is not part of the ration, and access to markets is limited.

Conclusions

Food security for new camp refugees is more fragile than for the old camp refugee population; this justifies maintaining a regular full general ration. Global acute malnutrition (GAM) is within acceptable levels, suggesting that specialised feeding programmes are not needed. Chronic malnutrition in Balochistan camps is a serious issue and needs to be urgently addressed.

Recommendations

Continue annual nutrition surveys to monitor the nutritional status of refugee population. Recruit/second a nutrition technical staff member to assist in developing and training of IPs in growth monitoring and promotion and in rolling out a monitoring tool.

115 Assefa F, 2001, op. cit. 116 National nutritional survey 2001-2002, op. cit. 117 Ibid. 118 Action Contre La Faim, 2003, Balochistan, op. cit. 119 Action Contre La Faim, 2003, NWFP, op. cit.

47 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Introduce “road to health” growth monitoring and promotion cards for all children under five. Terminate the SFP programme when the present food supply has been consumed, preferably when the winter is over. In the meantime, consider reintroducing the recommended standard ration. Address the present high levels of chronic malnutrition in Balochistan with a blended food ration as part of the general ration, particularly in Chaman camps (this may be another reason to move refugees from Chaman camps as water is probably a contributing factor). Supply training tools for identifying and recognising micronutrient deficiencies and to conduct training sessions on micronutrient deficiencies. Examine supporting local referral systems in the care of severe malnutrition. Consider food support to the chronically ill in the old camp population.

Lessons Learned

Since take-home food provided by SFPs is shared, rations should be increased to assist those suffering from moderate malnutrition to recover. Emergency feeding programmes such as SFP and TFP do not address chronic malnutrition. 127. Micronutrient deficiencies are difficult to diagnose and therefore are often missed, or only identified when there is a major crisis.

5.8 EPI and Child Health

128. The BHU is the main focus within a refugee camp and particularly with Afghan refugees, where there are extremely limited outlets where women can gather. In many instances, women may not even leave the compound to collect water. Women certainly do not have access to the market place.

129. In accordance with widespread cultural practices, many women do not leave the household for up to 40 days following delivery. The mother-in-law is a very powerful player in the household and often delivers the new babies. The Female Community Health Worker (FCHW) is instrumental in supporting the mother post- delivery and often brings the newborn to the BHU to check the birth weight and administer the initial vaccinations. ALL IPs encourage FCHWs to bring new infants to the BHU within three days of delivery.

130. There is a culture of tightly binding small infants with a cloth in order to: (i) prevent the infant from dislocating joints; (ii) to straighten the infants body; and (iii) prevent the infant from getting tired (‘too much moving makes the infant tired’), and the child settles better when bound.

131. Growth monitoring and nutrition education is conducted at the clinic targeting high-risk infants and malnourished children (see Chapter 5.7. on nutrition). Breast- feeding is the norm for at least four months, with weaning starting between 4-6 months.

48 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

132. Mortality and morbidity is recorded in the BHU statistics. The FCHW is mainly responsible for mortality data while morbidity data is calculated from clinic statistics.

133. EPI is one of the core activities within the BHU. The vaccinator is responsible for this and the cold chain facilities vary with some BHUs receiving vaccines from a central area on a daily basis while others have the refrigeration in the BHU.

EPI

134. In general HIS reports on EPI coverage appear to be good, but much of this data is flawed, particularly the population figures (recorded birth rates are low resulting in lower numbers of under one year). Mothers are well aware of the importance of vaccines. When asked, the mothers were able to list the vaccines had been given. This is particularly the case in the old camps, where the population has been exposed to health education over many years. Results from the nutrition surveys conducted in 2003 indicate measles vaccine coverage at 91% for NWFP, 75.4% at Chaman camps and 84% for Mohamed Khel camps.

135. Unfortunately, since HIS is so unreliable it is difficult to know the overall coverage. In fact according to the quarterly HIS data120 only 57% of children under one year of age received BCG in the period from July to September 2003. However, if one looks at the Balochistan HIS data121 for the month of September 2003 the coverage was 92.5%. It is possible that some of the data is missing from the quarterly report. It is also very difficult to calculate the coverage if the denominator is not accurate. The standard figures used are between 1.3-1.6 million refugees.

136. There is some confusion over the dosage of the BCG vaccine. Some health personnel stated that a different dose is given to children over one year, while others thought that children over one year should not receive the BCG vaccine if it had not been given earlier, since "they would have acquired natural immunity by then".

137. While HIS records indicate enormous vaccines wastage, in particular the BCG vaccine, it is not clear if the information is accurate. Both the quarterly and monthly HIS indicate that more than 50% of the vaccine is wasted, yet nothing has been done about this wastage. If it is the case then this is extremely costly and wasteful. BHUs have no standard timetables for vaccine provision. Some BHUs open a new vial for each child on any given clinic day, others vaccinate once a week or perhaps even less frequently. In NWFP, it was stated by PDH that a policy was made to ‘vaccinate at every opportunity’ which is unnecessary, as the population is static and lives relatively close to the BHU.

138. The normal number of vaccines in a BCG vial is twenty but vials with forty vaccines were imported at one point, a measure that could increase wastage. The wastage for measles vaccine is around 30%. In general, there are no clear, standard guidelines on vaccination procedures available to all BHUs.

139. Although the Polio Eradication campaign in both Pakistan and Afghanistan has been very intensive. Pakistan remains one of only three countries with reports of active polio cases. Information on Afghanistan is not current but EPI coverage is

120 National Quarterly HIS report (July-Sept, 2003). 121 Balochistan provincial analysis of HIS report September 2003.

49 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 poor, with routine polio campaign days. In particular, the border areas of Afghanistan/Pakistan have been targeted. The normal peak period for polio outbreaks is August to October, but so far this year, there have been no newly reported cases. In December, there are further polio campaigns planned in both Afghanistan and Pakistan.

Child Health

140. Curative care for children suffers from the same problems as curative care for adults, including an over emphasis on treatment with drugs, and a lack of proper clinical examination and diagnosis. Once again, the health staff is over-burdened with a large numbers of patients, many with minor complaints and the lack of a proper system of triage and guidelines. In Balochistan, in September, about one third of the patients seen were children under five years of age, whose main illnesses were chest infections and diarrhoea. This is similar to the quarterly HIS report. One BHU visited in Mohamed Khel only used tiny pieces of paper to write down patient details which normally only included patient name, sometimes the presenting problem and the drugs given. The pieces of paper were collected in a box but not filed which is very poor practice. Mortality records were not accurate.

141. In Asgaro camp, MC visited a few households. One mother had a sick child and had spent nearly 2000 Rs ($35) on care outside the camp, the second woman said she did not go to the BHU either but preferred to go to outside clinic, since she had to wait too long to be seen in the camp BHU. In other camps such as Thall, mothers were very satisfied with the health facilities.

142. Health education and promotion varies considerably among the different BHUs. Some IPs appear to have good training materials. There was a training course in progress during the evaluation visit. In general, the compounds visited were clean and mothers were aware of importance of child hygiene. Additionally, some mothers were aware of how to treat diarrhoea at home.

143. There is a high level of staff turnover particularly in the PDH system. In one area, there was apparently 60-70% staff turnover within the period of a year, according to Dr. Akbar, head of PDH Balochistan. Furthermore, the PDH staff rotation is organized centrally so that the supervising medical officer has no say in these movements. It is difficult to maintain standards, update skills, develop training courses and maintain staff morale with such high staff turnover.

Conclusion

EPI coverage in the camps is better than that in the host population or in Afghanistan. No major outbreaks of measles have been reported.

Recommendations

Develop basic standard guidelines for EPI procedures and coverage for all IPs, and a simple system for collecting relevant data as part of the HIS

50 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Check vaccine wastage and put procedures in place to reduce this waste. Ensure the procurement of standard doses of vaccines, involving other OPs such as UNICEF for this procurement process. Develop basic standard treatment guidelines for simple childhood illnesses and reducing the focus on curative care. Introduce triage for all curative care. Make sure that measles coverage reaches 90-95%. Measles outbreaks are extremely serious particularly in vulnerable communities; it is therefore essential that the coverage is high in order to prevent an outbreak. In areas where the population’s nutritional status is compromised, the threat is even more serious.

Lesson Learned

Lack of protocols and guidelines suggests that treatment is not standardized and may not be rational. Often, there is over-prescribing and polypharmacy is practiced.

5.9 Reproductive Health122

144. Reproductive Health (RH) services only began in the Afghan refugee camps five or six years ago. Afghan refugees in the Pakistan camps are mostly rural people who came from a demographic situation where both the mortality and birth rates were high. The population growth rate has been estimated to be 2.6% per annum.123,124 Health indicators in the refugee camps are generally better than those reported in either Afghanistan or rural Pakistan.125 Refugees have maintained a birth rate in the camps that is similar (42-43/1000) to that in rural Afghanistan, which has had the effect of increasing the population rate by 40-50% (to 3.8% per annum).126

145. The high population growth rate has had a number of untoward social and economic effects on refugees: many of which have been exacerbated by the camp conditions. During the initial emergency phase the maternal health situation in the camps was no better than that in rural Afghanistan. In the new camps, such conditions still prevail. The maternal mortality ratio in Afghanistan is one of the highest in the world and is reported to range from 1000-4000/100,000 live births:127 the effect of such high maternal death rates is compounded by high fertility. Afghan women have the highest risk in the world from dying from a pregnancy-related cause (lifetime risk of maternal death of 1:12 to 1:18).

122 See also annex 6 for field notes. 123 Reproductive Health Indicators & Outcomes among Refugee and Internally displaced person in Post- emergency phase camps, Hynes et al, JAMA, 7.8.2002, Vol. 288 No. 5. A most useful independent study. 124 Maternal Mortality among Afghan refugees in Pakistan, 1999-2000, Bartlett et al, Lancet, 23 Feb. 2003, Vol. 359 643-649. A most useful independent study on the area in Hangu that was visited during the evaluation. 125 Ibid. 126 Ibid. 127 Women's Commission for Refugee Women and Children, 2003, op. cit.

51 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

146. One of the most successful RH initiatives has been the establishment of community labour rooms (at Shamshatoo new camp BHU/PDH) and RH units that offer a wider range of RH services (Mohammed Khel new camp ARC, Hangu IRC, Haripur SCF). These units were found to be extremely clean, obviously well cared for, and patronized by the women in the community. Additionally, the local Pakistani population used some of the units (between 10-20% of the caseload were Pakistanis). In camps where there were labour rooms within one hour of the population, professionally supervised delivery rates were 20-30%, whereas in camps where no such facility were available, supervised delivery rates were < 5%. The competence and experience of the female doctors and ‘midwives’ working in the RH units and community labour rooms was quite variable, but all staff interviewed were obviously keen and enthusiastic about their work. None of the RH facilities was very busy and it would have been possible to double the client numbers without stressing the staff rota levels. Fees paid for services in the labour rooms and RH units cover about 25% of the expenses of the facilities. (Delivery fees were lowered from 300 to 150 Rs in May this year by order of the Commissioner. This has not resulted in any increase in facility use.)

147. Some of the old camps (e.g. Haripur camps) contained a number of private clinics run by Afghan refugee medical graduates. The RH advisor for SCF (Dr. Maqsooda Kasi) told GM that she has surveyed these Afghan doctors and has found that most of them are qualified and competent. She suggested that the private practitioners in the camps could be used to provide after hours coverage in the BHUs (see under ‘clinical care’ in Chapter 5.10.)

148. A number of NGOs have set up innovative and effective refugee RH programmes. These programmes could be easily and effectively extended to other areas. Many IPs find RH difficult and therefore do very little citing ‘cultural issues’ as an obstacle. The ARC programme in Mohammed Khel camp that provided health and domestic skills education to a group of girls aged 10 to 14 was particularly impressive. The quotation here is from Glen Mola’s Mohammed Khel camp evaluation field notes:

The female health workers of the ARC reproductive health centre, led by two Pakistani woman doctors, only began a peer group education project with young unmarried young girls since 1 December 2003. The parents were consulted and told that the group would provide the girls with an education on health matters, as well as give them instruction in cooking, sewing, etc.

I asked the ARC staff if they were planning to have peer group education also for the boys. They replied that this was their plan, but as they were all women, they started with the girls.

Although the group was advertised for 12-17 year olds, no 15-17 year olds were available (most of them are married), and the 10-12 year olds cried that they must be allowed to come too.

When I asked the ARC staff why they commenced the health education with a very difficult subject like HIV, they told me that this was at the request of the parents because they are very concerned about this mystery disease that they are hearing about in the media.

52 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

I engaged the girls in a quiz and discussion for about 30 minutes. They answered questions on how HIV was transmitted frankly and accurately. They could list all the means of transmission including sex, MTCT, re-used needles, blood transfusion — and even specifically mentioned homosexual as well as heterosexual intercourse.

When I asked how the virus was actually transmitted from one body to another during sex, they told me that the man put his fluid into the body of the woman when they had sex together and this was how the virus was transmitted. I inquired whether mosquitoes which drink our blood and then bite another victim could transmit the virus, a 10-year old told me “no”. Although she was not able to explain how transmission did not occur.

These girls appear very bright, eager to learn and quite unabashed by being asked about these things by a foreign male.

I asked the group if any were hoping to complete their education and then go on to vocational training for some profession. About half the class put up their hands and told me that they wanted to be teachers, doctors, nurses and the like. I advised them not to plan to get married too early if they wanted to fulfill their dreams. The ARC doctor then asked them if it would be possible to follow my advice, - they replied that they thought it would be!

This group discussion convinced me that it is actually possible to educate young Afghani girls on sexual and reproductive health matters.

149. Family planning (FP) is not being promoted at all in some camps, and even in those with well-developed FP programmes, it is being treated as a solution for excessive fertility, or only when a woman does not want to ‘have any more children’, rather than for routine birth spacing. Moreover, FP is usually only thought of and offered at strictly defined encounters such as the third post-partum check at 6 weeks (hardly any women turn up for this visit, and only 6% of those who booked antenatal visits attended the 6-week post-partum check in the Landi Karez new camp run by MSF where this policy is practiced).

150. Programmes that could easily be extended to other areas with extremely beneficial effects include:

- A programme for teaching girls aged between 10 to 14 years about health (including RH) as well as women’s skill operated by ARC in Mohammed Khel camp. The ARC health workers are planning to begin a similar education group for young boys soon. Community labour rooms and RH facilities are very successful in the camps and communities where they have been set up.

- Female community health workers (FCHWs) and trained birth attendants (TBAs) have been trained by a number of NGOs. These women are valued by their communities and are providing services to refugee women in camps. They also are the main source of demographic information for the health staff of the BHU (SCF FCHWs use a system of pictorial stickers to report on births, deaths, and other event). Some NGOs (SCF, DANESH, IRC, ARC) have developed excellent pictorial charts to explain RH processes (e.g. delivery planning) and to train illiterate women as FCHWs and TBAs.

53 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

- In this region, there is a high incidence of eclampsia, but only one NGO staff (ARC) is using MgSO4 to treat severe PET and eclampsia. Their eclampsia emergency kit could easily be reproduced elsewhere.

Conclusions

RH care should be a critical part of any emergency and post-emergency health programme. However, many IPs omit RH or only engage in the ‘easier’ parts of RH because they find it a difficult health area to implement. RH is a specialist area just like tuberculosis, malaria or Leishmaniasis control. RH specialists are usually necessary in order to facilitate successful introduction of RH programmes. A number of NGOs have developed some very innovative and successful RH programmes that could be easily extended to other areas

Recommendations

An RH technical support unit should be set up to assist IPs in developing and operating RH programmes. This unit could be formed within UNHCR Islamabad or set up by an NGO (or consortium of NGOs) with special interest in the RH area. There is an urgent need for tested and culturally-sensitive RH programmes, as well as community and health worker education programmes, protocols and services. Innovative and successful RH programmes and services need to be extended to other NGOs and camps. In areas where community labour rooms and RH units have been set up, it would be more efficient and effective to recommend that all women in the area use them for supervised births (rather than just those at ‘high risk’), antenatal and post- partum checks and family planning counselling and provision. Those providing health care to refugees and displaced persons need to follow the modern axiom that RH should be an integral part of family life, including all family health care services and encounters. Where there is a shortage of female medical staff in camp BHUs, women’s clinics could be set up by a NGO in the local district hospital to cater for referred women’s problems.

Lessons Learned

In order to reduce Maternal Mortality Rate (MMR) and prevent a dramatic acceleration in population growth rates in the refugee camps, it is necessary to introduce RH education for both men and women. This can be done by providing RH services at the onset of the operation, not waiting until the post- emergency phase.

54 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Cultural over-sensitivity has led to some IPs not being effective or doing very little in the RH area. Male involvement in reproductive health can lead to female empowerment in RH matters. It is possible to teach adolescent boys and girls about RH matter in the Afghan refugee situation. Community labour rooms and RH units are expensive to set up and operate and could be more effectively utilized by recommending universal supervision of birth and greater access to women of reproductive age for FP and other RH related activities. User fees at the level of Rs. 150-300 do not seem to deter refugees from using the labour rooms for supervised delivery. If an archaic approach to RH is followed, i.e. only offering RH assistance when the RH situation is dire, the uptake of RH assistance will be minimal and only have a small impact on women’s health. Because Afghan males have traditionally made all the decisions in RH matters for their families, RH initiatives for Afghan refugees need to emphasize male education and involvement. There is no choice without good information, nor is it possible to make good decisions without information.

5.10 Curative services - Clinical Care in the BHUs

151. Primary Health Care (PHC) is meant to be health promotion focused. However, all over the world, people seeking curative services inundate PHC facilities. In order to make PHC efficient and effective, it is essential that health managers recognize the conundrum of the curative demands of the community and their health promotion needs.

152. According to observations,128 health workers, particularly medical officers, spent the majority of their days consulting with clients about minor ailments. Some doctors are seeing more than 100 patients between 8.30am and 2:00 pm. In practice, this works out to less than three minutes per patient. It is simply not possible to see such a large number of patients in such a short space of time and provide good quality clinical care. Moreover, this level of service delivery means that important health promotional clinical activities such as health education, family planning, infant growth monitoring, and immunization status are left out.

153. Typically, patients were treated based on their presenting a complaint, i.e. minimal history was taken, virtually no examination was performed, and blood and urine tests were only occasionally conducted. Written records were even sparser. The presenting complaint was usually the diagnosis entered in the outpatients’ daily roll; usually no diagnosis at all was entered in the family health record book. In fact, the only writing in the patients’ record was usually the medicine to be given by the dispenser.

128 The comments and conclusions in this section are based upon visits by Glen Mola, to six BHUs (three PDH and three NGO operated), exit interviews conducted with more than 30 patients, and the review of 140 family health books in the various BHUs evaluated.

55 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

154. In most of the BHUs evaluated by the GM it was seen that patient flow was from the waiting area in front of the BHU into the record room where the family health cards were kept (this was often in the laboratory area). After getting the family health book, the patient went to wait in a queue for the doctor. From the doctor, patients were directed to the dispensary for oral drugs or vaccinator for an injection - sometimes via the laboratory. The nature of this patient flow (with the patient ending up in the dispensary) is that all patients ended up receiving drugs - indeed some patients received quite a number of different drugs. The family health book is retained by the dispenser who puts it back in the record system of the BHU at the end of the clinic.

155. There are no treatment protocols for the many minor ailments that account for the majority of cases each day. With one exception in Haripur (SCF US), there is no triage system or patient screening before entering the doctor’s consulting room. The supervisors of the BHUs (NGO medical directors and the field supervising medical officers of the PDH areas) are not able to provide any clinical supervision, as they are health managers and not clinicians.

156. In the middle of this year, the Commissioner for Afghan refugees in NWFP decided that NGOs should reduce the user fees charged to clients who attend BHUs for curative services from Rs. 20 or 30 down to Rs. 5. In the months following this decision, the numbers of clients attending for minor ailments and inconsequential medical problems increased by 30-35%. WHO recommends a frequency of 0.60 consultations per capita per year129 – UNHCR should monitor consultations per capita by looking at its own comprehensive yearly statistics.

Conclusions

BHU doctors are often overwhelmed by a very large caseload of minor ailments that leads to inefficiencies in the curative health care system. Afghan refugees usually judge the quality of their health care facility based on curative care factors. There was virtually no clinical care supervision in the BHUs. In the old camps, refugees expect to have to pay some small amount for good quality curative health services. It appears that in the absence of other meeting points for women and their children, they tend to use the BHU as meeting point. This can be very detrimental to curative care services. Many female health workers appear reluctant to work in the camps in Balochistan because of fear of the Taliban. Most infant deaths occur after-hours or on week-ends.

Recommendations

129 World Bank (1993), World Development Report 1993: Investing in Health. New York, Oxford University Press.

56 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

The recommended pocket book of standard management referred to in chapter Norms and Standards needs to have a section on how to quickly evaluate and deal with common minor ailments such as backache, and various other body aches and pains. IPs need to have a clinical supervisor who can assist them in setting up more efficient patient flow mechanisms in the BHUs, provide clinical supervision to the doctors working in the BHUs, and supply some in-service training from time to time. Patient flow through the BHU should begin with triage or client screening for those who are very ill and need priority attention. The lady health visitors (LVH) and nurses could conduct the screening. Training on the rational use of drugs is a priority since a great deal of money is wasted on unnecessary polypharmacy treatments of minor ailments. This initiative will need to be integrated into Afghan community education on quality clinical care and into the introduction of standard treatment protocols for the management of common conditions. This will avoid mismatch between quality care and community expectations130. Every camp needs a meeting spot like a community centre where people can find entertainment and do recreational sports. Even a television room/tent is recommended, where various groups can view appropriate programmes on a rotating basis. Not only would this facilitate education of various kinds but also it would provide an alternative place for women with children to meet. The savings on unnecessary drug usage would probably completely offset the expense of installing a television within 6 months. Where there are shortages of female health workers, the supervising NGO could enter into negotiations with the local MoH authorities to use space in the local civil hospital to conduct women’s clinics there a certain number of days per week. In areas where qualified Afghan refugee doctors operate private clinics in the camps, the NGOs and UNHCR could consider asking them to provide after- hours service to BHUs on a rotating basis. Payment for services could be made to the private doctor as usual, but the supply of initial doses of emergency medicine could be given by local arrangement.

Lessons Learned

A patient flow system that is oriented toward the drug supply will always end up providing the patients with too many medicines. Requiring a doctor to see more than 100 patients in five hours will lead to superficial clinical assessments, occasional misdiagnoses and overlooking the real problems of some seriously ill patients. NGO medical directors and PDH field supervising medical officers are usually not clinicians and as such are not able to provide any clinical supervision of the BHUs under their care and management.

130 A Swedish doctor was recently hired to conduct a study on the rational use of drugs with WHO Pakistan, but his report was not available at the time of our visit to Pakistan.

57 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

If there is nothing else to do or no place else to meet in a refugee camp, then a large number of people will go to the BHU as a social meeting point and once there seek unnecessary medical attention. Small user fees can serve to prevent ‘social over-usage’ of health services in the refugee camps. Different curative care systems and organization can lead to ineffective care or irrational polypharmacy.

5.11 Medical supplies

157. Following the GoP model of decentralization to the provinces purchases of essential drugs and medical supplies for the UNHCR health programme in Pakistan are handled at provincial level (donations only are received, stored and distributed from Islamabad). The budget allocated for procurement in the year 2003 was close to US $ 1,300,000.131,132 Purchases are made once a year by the provincial PDH who buys for all of the other IPs, who in turn are represented in the drug purchase committee. Drugs are then delivered quarterly from the central provincial warehouse to IPs and monthly to PDH’s own BHUs.

158. In the BHUs visited, there was a good control system in place, whereby all of the drugs dispensed are properly registered. Empty vials of expensive drugs (some antibiotics and anti-Leishmaniasis drugs) are kept to verify their use. Pharmacy stores and stock cards are generally well kept. Apart from the over-use of drugs, it is unlikely that a significant amount of wastage occurs at this level.

159. UNHCR has raised ‘some concerns over the controversial issue of the quality assurance of pharmaceutical products produced in Pakistan and considering the large number of manufacturers and the vast range of drugs available in the market’, they have suggested pre-qualifying suppliers prior to the tendering process.133 However, it appears that the GOP insists that UNHCR purchases at national and not international level.134

160. Decentralizing procurement of drugs and medical supplies is an inefficient procedure, whereby the tendering process, drug quality control and control of procedures needs to be executed three times instead of once, causing UNHCR to lose the opportunity to obtain better prices.

161. The biggest problem, however, is the lack of control that UNHCR and the IPs have over an ineffectual and non-transparent purchasing process. We have mentioned the financial irregularity documented in the external audit report in NWFP. A large part of the roughly US$ 360,000 not accounted for originates from irregularities in procurement: ‘...drugs purchased at rates higher than market price…purchase of items near to expiry dates… non-transparent tendering process

131 Mission report on revising pharmacy management guidelines in Pakistan UNHCR to propose appropriate solutions for the identification of shortcomings, 9-28 Sept. 2003, by Nabil Makki. 132 Strategy for procurement of medicines for Pakistan 2003, R.G. Elbro Sr. Regional Supply Officer, UNHCR, Islamabad. 21.1.2003. 133 Mission report on revising pharmacy management guidelines in Pakistan UNHCR to propose appropriate solutions for the identification of shortcomings, 9-28 Sept. 2003, by Nabil Makki 134 Strategy for Procurement of Medicines for Pakistan 2003, op. cit.

58 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 resulting in irregular purchase [nearly Rs. 5 million]… procurement of medicines in excess than requirement’.135 UNHCR could do better to channel its ever-scarcer resources toward proper use.

162. We understand that UNHCR is aware of these problems. Two missions have been looking into the Pharmacy Management System (PMS) in 2003, and have come up with somewhat different proposals. While one proposes that ‘drug purchase, (which usually costs on amount of Rs. 50 million will be handled by UNHCR itself,’136 the other has reservations. They said, ‘UNHCR does neither have the capacity to undertake this procurement nor the technical expertise (pharmacist) to assist with the technical evaluation and staff for the follow-up with the suppliers, deliveries, inspections etc.’ and goes on to suggest that ‘the best option to meet UNHCR’s needs for the procurement to be undertaken by the PMS [chaired by PDH]. However, UNHCR does have a monitoring role to play, which should be enhanced and defined further.137 In view of UNHCR’s previous failure in monitoring the drug procurement and the implication of PDH in procurement fund embezzlement, caution is required here.

Conclusions

Apart from the over-prescription of drugs, it is unlikely that a significant amount of wastage occurs at BHU level. Decentralizing procurement is an inefficient procedure, whereby the tendering process, drug quality control and control of procedures need to be executed three times instead of once, causing UNHCR to lose the opportunity to obtain better prices. Irregularities in the drug and medical supply procurement process by PDH results in a yearly financial loss of about US $ 200,000 for UNHCR, if not more.

Recommendations

The drug procurement should be put into UNHCR’s own hands and centralised in Islamabad. This represents an administrative and operational burden, but may make the procedure more effective, by providing better quality control, more efficiency, better prices, and less corruption. If the GoP insists, the purchases could continue to be made in Pakistan and not internationally. Pre-qualification of suppliers, prior to the tendering process, is certainly indicated. Hints can be taken from other organizations with a wealth of experience in this field, such as ICRC.

Lesson Learned

The procurement of drugs and medical supplies demands high standards for the quality: of the products purchased and for monitoring and supervision. If the

135 Certification audit report on the UNHCR funded projects of the Commissionerate for Afghan refugees NWFP, Peshawar 2002, Director Audit Federal government, 10 Fort Rd, Peshawar. This report is absolutely damning indictment of irregularities in financial management. 136 UNHCR Sub-office Peshawar. Health Sector Profile and Plans for the Year 2004. Public Health Officer UNHCR NWFP, 18 September 2003. 137 Strategy for Procurement of Medicines for Pakistan 2003, op. cit.

59 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

latter is not guaranteed, chances are high that things will get out of hand if left to a less-than-trustworthy IP.

5.12 Training

163. Training is an integral part of any programme. Training takes many forms, from day-to-day on the spot training by supervisors to formal sessions and courses. Over the years the IPs have developed training tailored to the needs of their staff and the beneficiaries. When dealing with different literacy levels among staff or beneficiaries, the training must be tailored to meet their needs and be appropriate for their level. They need the content of the training to empower them and not deter them from attending sessions. Trainings conducted by staff with the beneficiaries need to be well planned, varied and use visual tools, since the prevention of illness is one of the core principles of PHC.

164. When visiting the IPs it was evident that they were conducting different levels of training. International Medical Corps (IMC) has been doing training for the last four years, conducting courses every two weeks. To date, IMC has trained over 2,000 persons through semi-formal/formal training courses conducted in their training centre in Peshawar. IMC conducts the training for free, pays for travel expenses but does not pay per diems. Other IPs such as IRC have taken this training, but not the PDH, even though they have been invited to training courses.

165. In the IMC-supported BHU, there is also a trainer, who supervises staff. Additionally, she does on the job training with staff, particularly new staff. IMC has developed clinical guidelines for use in their BHUs. These were available in the clinic in New Shamshatoo.

166. Likewise, other NGOs such as SCF, IRC and DANESH have developed training materials, which are especially tailored for staff with low literacy levels. The material is also translated into the local language.

167. A group of TBAs (IMC and PDH) was interviewed at the new Shamshatoo camp. They were vocal and said that they were happy with the level of training. They seemed to be well informed about vaccines, infant feeding practices and hygiene. When asked about RH education with young girls, one TBA stated that ‘young girls do not need to know this information’. One of the reasons given for the low uptake of contraceptive was that ‘women like children a lot’. Visiting a household in Haripur MC met a TBA in her home. Although she was aware of child spacing, she did not want her daughters in law to use contraceptives. She wanted them to have as many children as possible: ten or twelve even if there were risks. Therefore, even when local women were trained it was difficult to change set habits.

168. Organizations such as HealthNet International (HNI) with a technical speciality have conducted training with all partners involved. This training has had positive feedback. Standards are in place, guidelines and monitoring tools. If standards drop, such as the level of false negative or positive malaria slide results then staff are sent on refresher courses. The staff members working in this area are happy with the level of training. There are similar training initiatives for TB and Leishmaniasis diagnosis and treatment with ARC.

60 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

PDH and training

169. Training in PDH is highly centralized, with seven master trainers based in Peshawar who only spend 12% of their time in the field when they are actually supposed to spend 75% of working time training in the field.138 Similarly, a female master trainer has been recruited but is based in the head office without a clear role. The PHC training course for community health workers (a 4-week course) is just a two-page list of subjects to be covered but no training material or tools have been developed for use during the course. The course is ‘in the heads’ of the trainers. One of the reasons given for not travelling to the field was the lack of vehicles and fuel, although there are reports of overuse of PDH vehicles for private use and overspending on fuel at the HQ level. It was stated by one IP that they conduct the training in Pashto and Dari, because some of the PDH staff only speak Urdu and this was also one reason why training uptake by PDH staff was very low.

170. At field level, the PDH Field Supporting Medical Officer (FSMO) felt powerless. Staff recruitment and transfers were decided centrally and transfers were frequent. There was no annual structured training plan and the field had no training budget. The FSMO had written to the DPH office and requested training for the following year, but by November this had not been done suggesting very little forward planning on either side. In 2003, in one agency (approximately one quarter of the total BHUs) a two-day HIS training, a three-day nutrition training and a one- week labour room staff training were conducted. This is totally unacceptable, given that staff turnover is so high. One staff member in a BHU complained bitterly about being moved three times in a very short period, with no choice or consultation. This is very disruptive for staff morale.

171. In general, no guidelines were in place in the PDH facilities, apart from some guidelines for RH, which were not being used at all. It was felt these guidelines were not very user friendly.

Budgetary Cuts

172. Training is often one of the first sectors cut when there is a reduction in funding. The UNHCR health budget will be substantially cut in 2004 therefore it is expected that training will be affected, which makes it essential that training is as streamlined and as efficient as possible. More coordination particularly between the NGOs and PDH is needed to conduct collective training. One NGO stated that PDH staff has been invited to attend training courses but does not attend. There needs to be collaboration in training and a transparent training plan from all IPs. A core group needs to be formed, with a lead agency putting together a plan of training needs, reviewing all trainings being conducted, training materials available and tools that will be used. Some very useful material has already been developed by some of the NGOs.

138 Consolidation of the PDH Structure. Proposals for the year 2004. Confidential report by UNHCR Public Health Officer, 27 October 2003.

61 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Conclusions

The training conducted by IPs — most of it by NGOs — is not very well co- ordinated. In the absence of standard guidelines, it is difficult to develop general training to be used by all stakeholders. This is costly and leads to duplication. Due to high staff turnover, training needs within the PDH are higher, yet the amount of training provided by PDH is considerably less than some other IPs. Some very good training materials have been developed especially for the Afghan refugee population that are culturally sensitive and in the appropriate language, that can used by other IPs.

Recommendations

Set up a core group to identify present training being conducted, training tools available and to streamline training according to needs. Improve on-the-job training at the field level with trainers based in the field Offer IPs and OPs with core competencies and successful RH programmes a lead role in conducting training in this ‘difficult’ health care area. Improve the recruitment process of trainers, employing personnel with relevant technical competencies; trainers also need to have appropriate language skills when training refugees. Base trainers within PDH in the field.

Lesson Learned

Budget costs often lead to substantial reduction in training: this is a false economy.

5.13 Security

173. In each office, security measures are coordinated by the UNHCR officers in charge of security, in conjunction with local authorities and UNDP. Despite the severe security problems just across the border in Afghanistan, and the widely suspected links of the Neo-Taliban and Al Qaida inside Pakistan, there are currently no severe security concerns for the UNHCR health programme. At times, visits to camps are cancelled due to curfews, especially in Kurram Agency.

5.14 Gender-related Issues

174. Afghanistan has a very conservative culture with most of its population living in rural areas. In general, women are housebound and the male is the main breadwinner and decision-maker. There are huge physical and social divides between the genders. Young boys and girls play together but from adolescence onwards; girls are often confined to their compounds. A large number of young girls who have started menstruating are married at the age of 13-14 years and sometimes as young as 8-10 years old. Marriages are generally organized and are often to older men. In times of economic stress, selling girls into marriage has been reported. In

62 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 general, men socialize outside the home with other males, and women stay in the home. Men and women do not socialize together.

175. Afghan adult literacy rates are low but female literacy rates are much lower: the female rate is estimated at 16% compared to 46% for male.139 Low levels of literacy add to the lack of empowerment for women, and affect the ability to recruit female medical staff. Additionally, many female community health workers are illiterate.

176. Due to this divide in society there is duplication of many services, this is particularly the case in health care where female patients will often only see female health workers (there are exceptions to this rule). In the community, female health workers can visit homes but male health workers cannot. In the old camps, there is only one medical officer per BHU and it is usually a male; therefore, many women end up being seen by the LHV instead of a medical officer. In some instances, a female MO rotates between a few BHUs.

177. Male and female health workers are trained separately. The health information collected by the different CHWs is completely separate as well. In the home women and men do not eat together. The men eat on their own and women and children eat together. The men are normally fed first which means often receiving the most nutritious food.

178. The gender divide affects RH education. Couples tend not to go for counselling/education together. In general, women appear to have more access to health education (particularly in RH), even though the men, (who do not get RH education) are the decision-makers in this area.

Women’s Access to Health Care

179. Women are the primary health care givers, looking after children at home and taking them to the clinic when they are sick. In many BHUs, men and women do not enter the same door to the clinic. More often children are accompanied to the clinic by their mother or by another female relative. Men have access to outside clinics so they often pay a little more instead of waiting and queuing at the BHU.

180. When there is no female MO, female patients are seen by the LHV, especially if it is a gynaecological related medical matter. Even in the BHU in Asgaro, although there was a female MO, the LHV diagnosed and treated all the STIs. When there is no FMO, the care that women receive is inferior, since less qualified health workers see them. However, women attend the clinic much more frequently than men do and they are more dependent on it for their medical care. With a total fertility rate of around 6-7, many visits will be pregnancy-related.

181. PDH maintained that it was difficult to recruit female medical staff, especially in rural areas. IPs in Balochistan voiced similar constraints. However, in Kurram Agency, where most of the MOs in the BHUs visited were men, local stakeholders held the opinion that female MOs could be found. In the Kohat district, they found a different solution. By rotating services, one FMO could cover several BHUs at the

139 UNICEF, 2003, op. cit.

63 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 same time. Innovative ways of recruiting and keeping female staff is extremely important, as their role is essential for good health care, especially for female refugees.

182. Dr Glen Mola took the lead in the RH part of the evaluation. There was a general perception (among UNHCR staff) that he would not have access to women at all. Some UNHCR staff members were taken aback when he expressed his wish to meet with female health workers and female patients. In fact, he got a surprising amount of access to these different groups, and the women were assertive and vocal. This showed that, there could be an overemphasis on “cultural sensitivity”, and that this could have a negative impact on programming. If male health workers are professional, there should be no reason for them not to have more access to women. Some RH procedures however, may remain out of bounds.

183. It was been reported that three female MOs are stationed at the PDH offices in Peshawar without any portfolios, at a time when a number of BHUs are without female doctors.140 There is consensus among all stakeholders that the male/female staffing ratio needs to be improved, but enough is not being done to encourage women to work in areas that are more isolated. UNHCR guidelines on the protection of refugee women recommend that they ‘provide incentives for the recruitment and retention of qualified female staff’.141 Once again, even though there are cultural issues around female staff travelling and staying in rural areas there are always ways around these issues.

Female Community Health Workers (FCHWs)

184. The FCHW’s role is important. She visits women in their home, particularly around delivery time. She often brings the newborn to the BHU for initial weighing and vaccines. The FCHW is responsible for tracing EPI and nutrition defaulters, and recording routine health information such as mortality data. Many of these women are illiterate and it was, therefore, important to give them simple tools to do their work. One IP had a system of recording data by using picture stickers, which seemed to work very well. The workload for FCHW seems to be much bigger than that of their male counterparts. However, in many cases there are twice as many male to female community health workers. There were 1,739 male and 865 female community health workers in the PDH in NWFP.

185. In general, the posts of male CHW seemed to be ‘political appointments’. The meetings with groups of male health workers did not prove very informative to this evaluation. In one instance, the male health worker had inherited his position from his father. At present, it appears that male CHWs shy away from any form of reproductive health; this is probably because they do not receive any serious amount of education/training on this topic. In the four-week training course designed by PDH for new community health workers, one day’s training is used for all the different systems of the body including RH, and another day is used for MCH/RH. This is all the time devoted to this complex and sensitive subject. There are no follow

140 Consolidation of the PDH Structure, 2004, op. cit. 141 UNHCR (2002) UNHCR Policy on Refugee Women and Guidelines on Their Protection: An Assessment on Ten Years of Implementation. An independent assessment by the Women’s Commission for Refugee Women and Children. UNHCR Geneva.

64 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 up refresher training courses. FCHWs appear to receive more on-the-job training, since they are more involved in the BHU.

Registration

186. In the BHUs, refugees are registered in family books, with the male head of household as the primary name in the family book. Women are not registered as single individuals unless they are heads of households (mostly widows). It is UNHCR policy, however, that all refugees should be recorded as individuals. UNHCR guidelines on the protection of refugee women recommend ‘improved registration mechanisms that allow each individual to obtain his or her own card’.142

Activities for Women

187. There are few opportunities for women to engage in any activity outside the home, and certainly none outside the camp. Although many of the camps have been in existence for a great number of years the main focus has remained on health care and disease prevention, rather than on the well-being of individuals. There have been attempts in some camps to provide other services for women, for example Inter SOS has skills training centres in Asgaro camp, including training in embroidery, tailoring, handcrafts and cooking and informal education such as language training.

188. However, income-generating activities for women are very limited. Some small initiatives are in place such as rope making in Kurram Agency, carpet making by Turkmen women in Balochistan. Older women in particular if they are widowed (the young would not be allowed by their families) may find employment in Pakistani or wealthier camp households. The lack of employment opportunities contributed to female subservience vis-à-vis their spouses. Women did not have access to markets and were therefore dependant on their men for the purchase of all their personal effects as well as household utensils and food. This lack of access and control was the main way of keeping one-half of the population subservient to the other.

Adult Education

189. Informal education and skills training for males and females is not apparent in most of the camps, but would be extremely beneficial. Inter SOS is involved in some training in Asgaro camp, and ARC has started adolescent education programmes in Mohammed Khel camp. It is important to offer similar services to both sexes particularly to gain acceptance among the male decision-makers of the benefits of these services. There may be fewer uptakes from the adult male population as men leave the camp to work.

190. Community centres can be used for many types of informal education. Brochures and information leaflets with health messages could be accessed through this type of facility. It may also be a useful forum to introduce education around sensitive topics such as the benefits and methods of child spacing and HIV/STI prevention. This type of facility is particularly beneficial where the camps are remote and employment opportunities are limited. Even a satellite television in the

142 Ibid.

65 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 community centre, would be a most useful adjunct to adult and child education. Unfortunately, services such as this are more likely to be stopped when there is a budget cut.

Conclusions

There is major gender inequality within the Afghan refugee population affecting the basic rights of women. Programmes need to be innovative and sensitive to address this inequality.

Recommendations

Actively recruit female staff to vacant health posts and address their operational difficulties for working in rural areas. Consider rotating FMO between a number of BHUs where recruitment of female health workers is not easy. Improve the ratio of female-to-male CHWs. Develop more innovative social programmes particularly for female refugees as an outlet to the BHUs, which appear to be the main/only meeting or socializing place. This should reduce UNHCR’s drugs expenses, especially since there is a widespread practice of over–prescribing.

5.15 Protection

191. The principal mandate of UNHCR is ‘providing international protection for refugees’. Secondly, UNHCR in conjunction with the governments of Afghanistan and Pakistan also has a major role in repatriation: ‘repatriation of Afghan citizens who have sought refuge in Pakistan shall only take place at their freely expressed wish based on their knowledge of the conditions relating to voluntary repatriation’.143

192. For over two-and-a-half decades, Pakistan has hosted a large number of refugees, well over two million at times and this has serious implications for the country in many respects. In some areas close to the border, the physical numbers of Afghan refugees is higher than the host population. This affects not only the services such as health, in particular the referral systems, but also the economy, since more people are competing for the same work, particularly daily labour. It can also have an impact on security.

193. Pakistan has not signed the 1951 convention for the rights of refugees, which would give refugees a “refugee status” and therefore more protection, rights and access to assistance. In Pakistan, the refugees are tolerated, but their situation is not legally recognised; they do not fall under Pakistani law, but remain in ‘legal limbo’, which makes UNHCR’s protection work more difficult.

143 Agreement between the Government of Islamic Republic of Pakistan, The Transitional Islamic State of Afghanistan and the United Nations High Commissioner for Refugees governing the repatriation of Afghan citizens living in Pakistan, 2002.

66 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

194. The “rights” accorded to refugees ebb and flow according to the politics of the host country. At a meeting with the deputy secretary of SAFRON, he stated ‘we do not apply Pakistani law in refugee camps’.

195. Economics play a big role and there is often overwhelming pressure by the host government to encourage refugees to go home. According to a PDH official in Balochistan, the pressure on the local infrastructure ‘causes resentment in the local population’. Over the years, there have been a number of initiatives to repatriate the refugees back to Afghanistan. In 2002, an estimated two million refugees returned home. There is presently strong pressure to repatriate as many as possible. It was hoped that 600,000 would go home in 2003, but by November 2003, the figure was only 340,000.

196. Whenever the GoP wishes to increase voluntary repatriation, this affects the day-to-day living for refugees: many refugees stated that over the last few months, harassment by the police has increased. Often movement is restricted, which affects the refugees economically. This is particularly serious when there is no food distribution going to the old camps or urban refugees. Some of the refugees in an urban settlement in Islamabad say they are always blamed if there is any trouble in the community. (e.g. an assault, rape, robbery, etc.). They are often locked up by the police and have to pay their way out. These refugees have little or no access to UNHCR protection officers, as they are part of the “invisible” refugee caseload.

Sex and Gender Based Violence (SGBV)

197. Protection for women is particularly important. Girls are still married off at very young ages, sometimes as young as 8-9 years old. Assaults and rape can be difficult to report, and such incidents are often kept quiet and not reported to authorities.

198. Identification and protection of women from SGBV, domestic violence, exploitation and early marriage is difficult, particularly in a very closed community where access is difficult. Barriers to refugee protection are ‘exacerbated for women and girls because of their unequal status in society’.144 The UNHCR protection officer in Balochistan reported 12 SGBV cases in Chaman camps in a six-month period in early 2003.145 Some IP staff felt that ‘there are no clear guidelines on how to deal with SGBV’, stating that reporting of cases is not sufficient. UNHCR and UNFPA developed guidelines in 1999, and have done some training since, but it appears the documents have not been well disseminated.

199. The primary way to target the issue of SGBV is through the BHU, with other women. It also depends if there is a skills-training programme in place in the camp. However, health staff is not well trained or informed on how to handle/counsel patients that present with SGBV. As it is a very sensitive issue, it is imperative that cases of SGBV are reported, in order to know the magnitude of the problem. UNHCR guidelines on the protection of refugee women recommend: ‘more vigorous use of national laws for enforcing protection and human rights’,146 but this is difficult when

144 UNHCR (2002), op. cit. 145 Women’s’ Commission for Refugee Women and Children, 2003, op. cit. 146 UNHCR (2002), op. cit.

67 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 local laws are not enforced by national authorities nor recognized in the camps by the refugees.

200. This is another reason to support an equal ratio of female-to-male community health workers both in the community and at the BHU. ‘The small number of female staff in place’ is just one reason why SGBV problems are not addressed sufficiently.147 UNHCR guidelines on the protection of refugee women recommends ‘Promotion of the multi-sectoral approach so that the refugee community, community services, health, protection, programme field and security staff can work together to prevent and remedy SGBV’.148

201. There is a need to develop more innovative types of programmes targeting females with the primary goal of empowerment. This can take the form of adult literacy classes and skills training. It is important that women have more outlets (other than just visits to the BHU, as is the case for many women in the camps at present). Visiting the Inter SOS skills centre in Asgaro was a very positive experience. The training available included the following languages, English, Dari and Pashto. The skills taught included drawing, cooking, tailoring, embroidery and handcrafts. Training was provided for 120 girls and 80 older women. Unfortunately, the programme was being phased out at the end of 2003, due to financial constraints. Such programmes also create an ideal environment to introduce educational topics such as human rights, identifying SGBV and issues around early marriages.

Conclusions

Refugee protection is a crosscutting issue that needs to be addressed at different levels. Empowerment of women is essential. The urban ‘invisible’ refugee population has little access to legal protection, as they are not recognized as refugees. Health-related protection issues concern mainly Sexual and Gender-Based Violence (SGBV), rape and forced under-age marriage.

Recommendations

Improve the balance of female health workers at all levels. Systematically gather data on SGBV, in order to know the magnitude of the problems and to explore potential ways to tackling them in cooperation with local partners. Develop short training courses tailored for staff at different levels, with both OPs and IPs, in order to raise awareness about protection issues and procedures/protocols.

Lessons Learned

Budgets cuts often result in axing social programmes such as the adult literacy and skills training. This has a negative impact especially on women and

147 Ibid. 148 Ibid.

68 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

represents a ‘false economy’, as social programmes are targeting the least empowered groups. It probably also leads to an increase in BHU attendance, as BHUs becomes the only outlet for women’s social lives.

5.16 Referral

202. By and large, the PHC service provided by the BHUs in the refugee camps gives better service than those available to the surrounding rural communities. However, if refugees require more sophisticated care than is available at the camp BHU, they must accept a referral to a district hospital or tertiary referral centre149 of the MoH. Only a very small proportion of refugees are able to afford the fees of private medical facilities.

203. The evaluators visited eight referral hospitals in Balochistan and NWFP, which are receiving many Afghan refugees as both outpatients and inpatients. Hospital administrators and doctors in the MoH hospitals at the border districts universally claimed that refugees were flooding their institutions to the detriment of the local people.

204. Chaman Hospital is a special case. It is a referral hospital in Balochistan where four new camps are situated near the Afghan border. WHO, Plan Pakistan and UNHCR have spent a great deal of money to upgrade this facility, however, it appears, that the institution is functioning at a low level and is grossly under- utilized. It was virtually vacant at the time of the evaluation because, for the past month (since 17 November 2003), there had been no fuel available to keep the patients from freezing overnight.

205. Two NGOs (PRCS in NWFP, AMDA in Balochistan) concentrate their efforts on trying to facilitate referrals for refugees to secondary and tertiary health facilities. They also provide food, payment for laboratory, drug and other fees in an attempt to assist refugees in the treatment of their conditions.150 A third NGO, Plan Pakistan in Chaman, provides an ambulance service to Chaman and Quetta, with daily presence in each camp and night standby in Chaman. A detailed evaluation of the services provided was not possible. However, the service provided by AMDA appeared not to be very cost-effective, using many resources (both in terms of staff and in terms of refugee assistance) in facilitating referrals to a mostly ineffective secondary and tertiary health care service. Respondents among BHU staff and refugees never mentioned incidences where patients would ‘get stuck’ trying to obtain services. By and large, admission to referral hospitals appeared to be always possible.

206. A great many refugees bypass the BHUs altogether: UNHCR field officers and the refugees themselves related that BHU services, particularly those run by the PDH, were accessible for such limited periods that refugees needed to go to Pakistan health facilities (both public and private) for health care. Therefore, the official referral figures (for instance, 179 patients to tertiary hospitals in NWFP in 2002) are certainly the result of under-reporting. Especially when taking into account that referrals of urban refugees are not computed. In NWFP, both camp and urban

149 In Peshawar: Hyderabad Medical Complex, Lady Reading Hospital and Khyber Teaching Hospital; in Quetta: Bolan Medical Complex. 150 PDH in NWFP also keeps a medical referral officer, whereas work is carried out by PRCS (source: Consolidation of the PDH Structure, Proposals for the year 2004. op. cit.)

69 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 refugees who manage to get access to charity hospitals151 (‘you have to know someone in there…’) prefer these for self-referrals. Charity hospitals are not free, but are less expensive than private hospitals.

Conclusions

By and large, admission to a referral hospital appeared to be possible for Afghan refugees; access is comparable to that of the poor local population. A large proportion of in- and outpatients of MoH hospitals in areas where there is a concentration of refugees are Afghans. A great many refugees bypass the BHUs altogether: (all urban refugees and camp refugees after BHU closing time), or of their own volition. Many resources (in terms of both staff and refugee assistance) are used in facilitating referrals to a mostly ineffective secondary and tertiary health care service. The referral structure at Chaman Hospital for the new camps in Balochistan is functioning at a low level and is grossly under-utilized.

Recommendations

Commission a local evaluation on the functioning of Chaman Hospital and share findings with the main stakeholders (including the locally active NGOs) in order to establish a strategy of revitalization for as long as the camps exist. NGOs involved in the referral process should contribute to extending clinical care at the secondary care hospital level rather than just spending considerable resources on merely acting as administrators and facilitators of the referral process. Not only would this relieve the client load at the civil hospitals, it would also serve to promote better relationships between MoH facilities and staff, and the primary health care providers and their refugee clients.

Lesson Learned

Massive investment in infrastructure and equipment of Chaman Hospital and leaving it to the MoH ‘to get on with it’ does not guarantee the provision of adequate secondary clinical (district hospital level) services.

151 Mercy, Al Jihad, ‘French Hospital’

70

6. EVALUATION CRITERIA

207. In this last section of the evaluation report, certain findings are highlighted — most of which were already presented — in light of the evaluation criteria specified in the TOR (Annex 3).

6.1 Impact

208. The UNHCR health programme has had a deep and lasting impact on the camp refugee population. The impact is, in good part, directly attributable to the health programme, and can be found or even measured by the four dimensions listed below:

Accessibility of PHC services; Improvement of health indicators; User satisfaction; and Improved health-seeking behaviour. 209. Camp refugees have better access to curative and preventative PHC services than the local or the non-displaced population. Health gains through PHC services are the basic assumption of the 1978 WHO Alma Ata declaration, of health gains through PHC services have been substantiated for Afghan camp refugees in the literature.152 Improving the HIS, as recommended in chapter 5.4 should further contribute to the document's impact. An unintended impact of the PHC programme was the increase in the rate of population increase by 40-50%: this is mentioned in Chapter 5.9.

210. User satisfaction was addressed throughout in this evaluation. While on the one hand, most respondents expressed general satisfaction with ‘their’ BHU, others gave mainly long waiting times as a reason to seek care in the private sector. The high use —even overuse — of services (see Chapter 5.10), however, can be taken as a proxy for at least sufficient user satisfaction: ‘people will not accept poor-quality services uncritically just because they are there’.153

211. The health-seeking behaviour of refugees (‘Afghans are more demanding than Pakistanis’) has a multiplier effect in the sense that it is transferred to the second and third generation and potentially more for the future, when/if returnees will seek and demand health care in Afghanistan. Mothers were able to list the vaccines given to their children, comment positively on use of contraceptives (‘we didn’t know about these things in Afghanistan’), and were likely to purchase impregnated bed nets.

212. Distributional effects have very much diluted the impact of the PHC programme. Urban refugees - as often, the poorer and less articulate – have been excluded (see Chapter 6.7). In Chapter 5.15 we also pointed out a missed opportunity for the health programme to have greater impact on women with regard to SGBV.

152 Bartlett et al, Lancet, 2003, op. cit. 153 Cassels A (1995) Health sector reform: key issues in less developed countries. Journal of International Development, 7 (3): 329-347.

71 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Conclusion

The UNHCR health programme has had a deep and lasting impact on the camp refugee population.

Lesson Learned

Distributional effects can significantly tarnish the success of an otherwise successful programme: urban refugees have been excluded.

6.2 Relevance

213. The UNHCR health programme as stated in its overall goal154 is extremely relevant for the refugees. More than two decades after the inception of the programme, it is difficult to find out whether and what kind of needs assessments were initially undertaken, but the application of a ‘blueprint approach’ for typical situations of concentration of refugee populations is justified. The provision of PHC services still corresponds in their present situation to a real and expressed need of the refugee population, and is certainly one of the best uses for UNHCR funds.

214. No major moving of goalposts with regard to the original overall objective has been documented, nor was it probably indicated. Authorities dealing with health in particular and refugees in general, repeatedly voiced a request for separate second and third-line health services for the refugees, who now put a considerable strain on the MoH hospitals as described in 5.16. In view of the lack of official recognition of at least half of the refugees (the urban ones) and the GoP’s de facto recognition of the camp refugees as temporary settlers, it is difficult to imagine that the GoP would have consented to establish separate referral structures.

215. It can be argued, however, that only ceteris paribus, that there was no need for a significant change of the overall objective. The refugee situation, however, did not remain equal. It changed dramatically after the post-1995 exodus into the cities (see Chapter 5.2.), with consequences described below, under ‘coverage’ (chapter 6.7.). It also changed for the refugees remaining in camps, by becoming perennial. At present, a third generation of Afghans refugees is being born in Pakistan. Sustainability has become imperative as described in chapter on assistance strategy (see Chapter 5.3).

Lesson Learned

With regard to these important changes in the refugee situation, a ‘process approach’ would have been indicated, rather than a continuation of the initial ‘blueprint approach’.

154 Goal 3: Refugees who arrived in 2000 and 2001 and reside in the new sites, benefit from basic assistance, including food distribution and domestic items, while refugees who arrived earlier and reside in refugee villages have access to community-based health care, primary education and water (Consolidation of the PDH Structure. Proposals for the year 2004, op. cit.).

72 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

6.3 Appropriateness and Participation

216. The health programme has generally been implemented in an appropriate way, taking customs and practices of the refugee population into account. One exception is the lack — in certain geographical areas and certain IPs —of female health staff described in the chapter on gender-related issues (Chapter 5.14).155 On the other hand, we have argued in Chapter 5.9 that ‘cultural over-sensitivity’ with regard to RH-related subjects can lead to missed opportunities. It is critical to tackle the RH issue with young girls and men.

217. It was not possible to gather in-depth, first-hand information about community participation, apart from taking note of its existence, and consulting with the health committees156. The fact that UNHCR field officers attribute ‘good community participation’ — without being prompted — to some IPs can lead to the conclusion that for other IPs, there is room for improvement. In the chapter on assistance strategy (Chapter 5.3), we have pointed out the virtual lack of participation by health service users, after an attempt to increase user fees between 1996 and 2002. This effort must be revived. Increases in material participation require increases in participation in the decision-making process.

Conclusions

The health programme has generally been implemented in an appropriate way, taking customs and practices of the refugee population into account. Exceptions are the lack of female health staff in certain areas and ‘cultural over-sensitivity’ with regard to RH-related subjects, which can lead to missed opportunities. Community participation, however, especially in the material sense, is minimal.

Lesson Learned

‘Cultural over-sensitivity’ with regard to RH-related subjects can lead to missed opportunities: e.g., RH is not tackled with young girls and men.

6.4 Effectiveness

218. UNHCR’s health programme has been effective for the camp refugees, if measured at the level of output indicators, consultations given, children immunised etc. The evaluators were not provided with comprehensive statistics, but anecdotal evidence from IPs, which leads to the conclusion that they all keep detailed statistics of their own activities. The occasional crosscutting survey produces useful data on effectiveness, such as the 2002 Survey Report on RH and EPI in Balochistan. A comparison, between ANC services attended by expectant mothers in old and new camps during their last pregnancy showed figures of 63.6% and 37.5%, respectively. Current use of family planning methods was found to be 12.2% and 6.7%. Lastly, the number of fully immunised children was 54.3% and 44.1%.157 More important than

155 Not everywhere and of all IPs; some actually excel in tilting the gender balance toward female staff 156 As mentioned in chapter 4, a strong caveat is indicated regarding the representativeness of these committees. One UNHCR field officer reported that ‘the same village elders sit in all committees. 157 Survey Report, Reproductive Health and EPI Coverage in Refugee Camps of Balochistan. December 2002. In collaboration with the technical assistance of WHO Quetta.

73 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 the absolute figures (there is always room for improvement) is the upward trend over time between old and new camps, which can safely be interpreted as a sign of effectiveness.

219. The current HIS is not able to provide reliable data on effectiveness (Chapter 5.4). Only with accurate and comprehensive reporting of data is it possible to audit performance and to monitor the effectiveness of UNHCR’s health programmes.

220. The indicators available to measure effectiveness are input, process and output indicators (e.g. number of staff, training provided and patients seen, respectively). Measuring outcome, such as treatment success, is notoriously more difficult. The above report,158 however, produced results on behavioural change, such as the cited use of family planning methods. The evaluators themselves repeatedly inquired into knowledge of CHWs, on HIV/AIDS for instance (see Chapter 5.9), from which knowledge gained (another outcome indicator) could be inferred by means of available controls.

221. A serious shortcoming with regard to effectiveness is the crosscutting gender issue, which was mentioned in Chapter 5.14. In the old camps where there is only one medical officer per BHU - usually a male - women with gynaecological complaints end up being seen by an LHV instead of a medical officer.

222. In the chapter on coordination (Chapter 5.5), we stated that there was a significant amount of leeway available for improving coordination. Certain gaps and overlaps were observed; for example, PDH did not fulfill its coordination role, superior and innovative health initiatives designed and carried out by some NGOs were not being extended outside their areas of geographical responsibility, etc.

Conclusion

The upward trend over time of certain outcome indicators in Balochistan, i.e. the difference observed between the new and old camps can be taken as a proxy for effectiveness of the health programme.

Recommendation

Measure effectiveness not only in input, process and outputs indicators (e.g. number of staff, training provided and patients seen), but also make an effort to more systematically measure health outcomes, such as treatment success, behavioural change and knowledge gains.

6.5 Connectedness and integration

223. The single most important issue in this regard is the short time horizon for planning. In Chapter 2, we have repeated the near-unanimous opinion of all respondents, that the assumption that Afghan refugees will soon return to Afghanistan is not a realistic one. In the end, there will be no PDH and no significant UNHCR or NGO assistance. Of the current stakeholders of the health programme

158 Pakistan Programme. UNHCR printout, unspecified.

74 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 only the Afghan communities and the MoH will remain. Neither of these remaining stakeholders, however, is currently involved in the PHC programme. In chapter 5.3, on assistance strategy, we have made the recommendation of involving both to a much higher degree.

224. Anecdotal evidence — not more — was found of local integration at different levels. One example is the use of camp BHUs by the local population. Most of it, however, appears to concern the treatment of Leishmaniasis, which is expensive and not available in Pakistan through the MoH services.159 In general, however, camp BHUs are frequented almost exclusively by Afghan patients; obvious geographical, but also other reasons were cited: ‘we discourage [the local population], tell them this [BHU] is for Afghans’. The local population also buys impregnated bed nets. Which have the additional advantage of being carried back to Afghanistan by those who repatriate. Lastly, an interesting example of integration is the IP MSF (Holland) which runs BHUs both in Chaman and in camps across the border in Afghanistan in the area of Spin Boldak; the latter camps are populated with IDPs who have fled the same drought and war as the refugees, but got stranded on the other side of the border.

225. On the other hand, we have seen a few obvious examples of missed opportunities for integration. PDH, for instance, appears to purchase contraceptives on the open market instead of using highly subsidised GoP supplies. Another example of a missed opportunity is the absence of UNFPA’s inputs into the UNHCR Health Programme (Chapter 5.5). Lastly, little evidence was found of participation of IPs/PDH in MoH training and vice versa.

Conclusion

In the long run, of the current stakeholders of the PHC programme only the Afghan communities and the MoH will remain. However, neither of these remaining stakeholders is currently involved in the health programme.

Recommendation

UNHCR must discuss the current and future impact of budget cuts in a transparent way with IPs and donors, and look at alternative, long-term assistance strategies.

6.6 Coherence

226. In the chapters on health policy and assistance strategy (Chapters 5.2 and 5.3), we have argued that the assistance strategy employed, i.e., the gradual decrease of funding of mostly foreign or purpose-created IPs, appears not to be embedded in or supported by an explicit overall policy of UNHCR toward the Afghan refugees. In the chapter on the context (Chapter 2), we have determined that formulating such a policy for a situation of transmigration would be difficult.

227. What course is UNHCR steering? The assumption that IPs can offset UNHCR budget cuts with bilateral funds is a shaky one, even if at present it appears that

159 Forty percent of Leishamaniasis patients are locals.

75 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003 international NGOs have sufficient funds to go around. In Chapters 5.3 and 5.12 we have pointed out the consequences of budget cuts. Next year, there will be more, and some IPs will reach their breaking point - some already feel insecure.

228. Is there an unwritten agenda to use service cuts as an additional incentive for voluntary repatriation? The example of over one million Afghans who live without basic health or education services in cities in Pakistan proves that such a strategy is only marginally effective. If no such agenda exists and UNHCR is only ceding to donor and host country pressure, the impact of these budget cuts must at least be foreseen and considered and a radical change in assistance strategy made with a much longer time horizon (as suggested in the chapter above and outlined in chapter 5.3). The current model of assistance will one day become obsolete – what will replace it?

Conclusion

The assistance strategy employed (funding of mostly foreign or purpose-created IPs, with progressive budget cuts) appears not embedded in or supported by, an explicit overall policy of UNHCR toward the Afghan refugees.

Recommendations

Be more transparent with IPs, donors and refugees about the role of UNHCR’s health programme in its overall policy. Consider and plan for the impact of the progressive budget cuts.

Lesson Learned

Decrease in service provision is only marginally effective as an incentive for voluntary repatriation.

6.7 Coverage

229. In the chapters on the refugees and their health and health policy (Chapters 5.1 and 5.2), we have pointed out the striking difference between the camp refugees who benefit from GoP recognition and basic UNHCR health and education services, and those who do not benefit from either. We have also argued that these urban refugees — apart from a small wealthy middle class — represent a negative selection, especially when food assistance to the camps was discontinued in 1995. Those who did not have an economic activity that allowed them to make ends meet left the camps in pursuit of economic survival in urban centres.160

230. The substandard living conditions of the urban refugees are described in an several publications.161,162,163 The fact that most of the Afghans who repatriated in 2002 came from urban centres and not camps indicates a personal judgement about the living conditions in Pakistan as compared to the uncertainties in Afghanistan.

160 A number of them via Afghanistan, where they decided not to stay 161 Refugee Reports, 2001, op. cit. 162 UNHCR, 2002, op. cit. 163 Women’s Commission for Refugee Women and Children, 2002, op. cit.

76 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

The recent wave of repatriation has come to a halt. The remaining urban refugees continue to be the subject of frequent police harassment while they try to make a living as unskilled labourers and their children scramble on garbage heaps trying to find recyclable goods and grow up uneducated.

231. Some efforts were made by others to cover the health needs of the urban refugees. In Quetta, Mercy Corps funds local NGOs to provide PHC services and in Peshawar, some charity hospitals still exist. However, these efforts are highly insufficient; the urban refugees rely on private practice and tertiary referral hospitals of the MoH for curative services and have hardly any access to preventative health services. It is difficult to explain UNHCR’s ‘out of sight, out of mind’ attitude vis-à- vis its less favoured clients. Perhaps it is because of pressure and lack of donor support from the GoP. UNHCR staff members usually give the latter as an explanation.

232. Apart from this —admittedly huge — gap in coverage, UNHCR’s de facto target population, the camp refugees, appears well covered.

Conclusion

While roughly half the refugee population, i.e. camp refugees, benefit from reasonably good PHC service, the other half, urban refugees, do not benefit at all.

Recommendation

Include urban refugees in UNHCR’s health policy framework.164

6.8 Efficiency

233. The efficiency of UNHCR’s health programmes leaves a great deal to be desired (more so even than the deficits in effectiveness). The most significant efficiency gains can probably be made in UNHCR’s technical/managerial capacity and the choice of IPs.

234. At the time of the evaluation, UNHCR's Health Manager in Islamabad was in charge of a US$ 6.8 million health programme and operations involving more than a dozen IPs with a total staff of well over 2,000. The lack of additional managerial presence in Quetta and Peshawar was deeply felt by UNHCR line management and IPs alike. In Quetta, health coordination meetings were no longer held. The position of health manager in Peshawar exists, but has only been intermittently filled in recent times. Considering that roughly 80% of UNHCR’s activities are concentrated in NWFP, a strong operational presence is needed with a senior public health person. This should probably be an expatriate, since the person is likely to be under a lot pressure. The Quetta operations are smaller and coordination among IPs is easier. A focal point for health, however, is still needed. By improving its technical/managerial capacity, UNHCR will be able to put its resources to better use.

164 UNHCR staff in Islamabad point out that this recommendation might be irrelevant because of the gradual scaling down of assistance.

77 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

235. Better value for money can also be gained though a better choice of IPs. Table 3 presents an overview of budget and staff per Health Unit of all IPs that have no co- funding.

Table 3: Budget and staff per health unit for IPs without co-funding

Implementing Partner Number of Budget per Health Unit Staff per Health Units (US $) pa Health Unit PHD NWFP 36 59,221 21 PDH Balochistana 15 40,518 16 Kuwait Joint Relief 8 31,320 15 Committee Union Aid for Afghan 9 25,204 14 Refugees Frontier Primary Health 3 27,326 13 Care Source: UNHCR, Islamabad (see Annex 4 for more information) Notes: aDiscontinued as from 2004

236. Owing to a lack of information about co-funding, it is not possible to calculate the same figures for the other IPs; those presented in Table 3 support the conclusion that NGOs do work more efficiently than the PDH. This finding can probably be extrapolated for and/or verified with, NGOs with co-funding. A certain allowance may be made for the co-ordinating function of PDH (albeit judged ineffective by most, see Chapter 5.5.). This allowance should be balanced by an economy of scale, considering the much higher number of health units PDH runs. Financial irregularities perpetrated by PDH (see chapter 5.3.) add to the inefficiency of their work.

237. The inefficiency of PDH can, to a large part, be attributed to overstaffing, the high number of senior PDH administrative staff has been mentioned (Chapter 5.3.), the duplication with PRCS for patients referral (Chapter 5.16.), and the fact that there appear to be FMOs stationed in Peshawar ‘without portfolio’ (Chapter 5.14.). Another staff-related factor contributing to PDH inefficiency is that trainers allegedly do not travel frequently to the districts nor do any substantial training (Chapter 5.12).

238. Efficiency gains can also be made in a number of other fields, already mentioned in the respective chapters, by:

Rationalizing BHUs (e.g., there is one BHU for 5184 refugees in Kurram agency);165 Reducing the alleged high vaccine wastage (Chapters 5.4, 5.8); Improving the gender ratio of CHWs and Medical Officers (Chapter 5.14.); Refraining from putting a significant amount of resources into an ineffective HIS (Chapter 5.4.); Improving the RH activities by strengthening technical capacity (Chapter 5.9.); Reducing the caseload of minor ailments in the BHUs (Chapter 5.10.); Reducing the significant drug wastage through over-medication (Chapter 5.10.);

165 Nobody’s fault, apparently, but due to a shrinking camp population owing to resettlement.

78 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Reducing staff rotation in PDH and thus improving staff morale (chapter 5.3.)

Conclusions

The sample of NGOs who do not have co-funding work significantly more efficiently than PDH with regard to staff and budget per health unit. This finding can probably be extrapolated to NGOs with co-funding. With only one health coordinator based in Islamabad, UNHCR is grossly understaffed at the health managerial level.

Recommendations

To strengthen UNHCR’s technical/managerial capacity by filling the position in Peshawar with a senior public health person, preferably an expatriate with a 3- year mandate. A focal point for health is also needed in Quetta166. To improve efficiency in the areas of: Rationalizing BHUs (but with care not to over-rationalize); Reducing the alleged high vaccine wastage; Improving the gender ratio of CHWs and Medical Officers; Cease supporting ineffective HIS; Improving the disjointed approach to RH by strengthening technical capacity; Reducing the significant drug wastage through over-medication; and To include efficiency indicators in routine monitoring.

Lesson Learned

‘What is not monitored cannot be evaluated’, the monitoring of efficiency indicators such as unit cost (e.g. per BHU, per consultation) is an important part of the evaluation of IP performance.

166 UNHCR staff in Islamabad point out that ‘expatriate focal points for health activities are being hired through UNV to strengthen UNHCR team's capacity in coordinating the health assistance activities and so in overseeing the plan to entirely reform and restructure the PDH’.

79

Annex 1: Bibliography

Documents and references read and perused regarding this evaluation: with some comments on linkage and findings concurrent with this evaluation.

1. Official Reports by Government and Consultancy Mission Reports

2002 Annual report of the Commissionerate for Refugees in NWFP, Brig (ret) Musthtaq Alizai. This report is very anecdotal and opinionated, contains virtually no budgetary or financial information, and many of the opinions are not substantiated by our evaluation.

Certification audit report on the UNHCR funded projects of the Commissionerate for Afghan refugees NWFP, Peshawar 2002, Director Audit Federal government, 10 Fort Rd, Peshawar. This report is absolutely damning indictment of irregularities in financial management

Mission report on Revising Pharmacy management guidelines in Pakistan UNHCR to propose appropriate solutions for the identification of shortcomings, 9-28 Sept. 2003, by Nabil Makki.

Agenda of Tripartite Commission Meeting at Dubai, 3.12.2003, chaired by UNHCR.

Role and Charter of duties of Medical coordinator UNHCR / CCAR, Islamabad, Pakistan.

2. Independent Research Papers and Texts.

Reproductive Health Indicators & Outcomes among Refugee and Internally displaced person in Post-emergency phase camps, Hynes et al, JAMA, 7.8.2002, Vol. 288 No. 5. A most useful independent study.

Maternal Mortality among Afghan refugees in Pakistan, 1999-2000, Bartlett et al, Lancet, 23.2.2003, Vol. 359 643-649. A most useful independent study on the area in Hangu that was visited during the evaluation.

A study of reproductive tract disorders among Afghan Refugees, Owoso E, march 1999 of Reproductive Health for Refugees Consortium for Save the Children Fund (US) Haripur, Some good information but some strange technical conclusions about RHIs.

Tomeczk B. et al (2000) Finds from a Reproductive Health Survey of Afghan Refugees in Pakistan. A collaboration between CDC and IRC.

King MH, Mola GDL, Thornton J, Bullough C, Guillebaud (2003) ‘Primary Mothercare and Population’, J, Spiegel press UK. (Available from [email protected])

Pocket Manual of Standard Treatments of Common illnesses in children for doctors and nurses in Papua New Guinea, PNG government printer 2003, 8th edition.

Pocket Manual of Standard Treatment of common problems in Obstetrics & Gynaecology for doctors and nurses in Papua New Guinea, 4th edition, 2000.

King MH, Savage, F, Bullough C (2002) Primary Childcare, WHO publication.

81 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Haleem I (2003) Ethnic and sectarian violence and the propensity towards pretorianism in Pakistan. Third World Quarterly, 24 (3): 463-477.

Closed Door Policy. Afghani Refugees in Pakistan and Iran. Human Right Watch Report , Vol. 14 No. 2 (G), February 2002.

All Our Hopes are Crushed. Violence and Repression in Western Afghanistan. Human Right Watch Report, Vol. 14 No. 7 (C), November 2002.

Refugee Reports, Vol. 22, No 7 (2001), by Hiram Ruiz. http://www.refugees.org/world/articles/Afghan_rr01_7.htm

Rowland M et al (2002) Afghani refugees and the temporal and special distribution of malaria in Pakistan. Social Science & Medicine 55 (2002) 2061-2072.

Chimni B (1999) From Resettlement to Involuntary Repatriation: towards a critical history of durable solutions to refugee problems. Evaluation of Humanitarian Assistance in Emergency Situations UNHCR/EPAU New Issues in Refugee Research. Working Paper No 2 [UNHCR durable solutions].

Crisp J (1999a) Who has counted the refugees? UNHCR and the Politics of Numbers. New Issues in Refugee Research. Working Paper No. 12 [UNHCR counting refugees].

Crisp J (1999b) Policy challenges of the new diasporas: migrant networks and their impact on asylum flows and refugees. UNHCR, New Issues in Refugee Research. Working Paper No. 2.

Jacobsen K, Landau L (2003) Researching refugees: some methodological and ethical considerations in social science and forced migration. UNHCR Evaluation and Policy Analysis Unit, Working Paper No. 90 [Researching Refugees].

Macrae J (2002) The bilateralisation of humanitarian response: implications for UNHCR. UNHCR EPAU/2002/15 December 2002.

Turton D, Marsden P (2002) Taking Refugees for a Ride? The politics of refugee return to Afghanistan. Afghanistan Research and Evaluation Unit. [UNHCR Afghanistan].

UNHCR (2002) UNHCR Policy on Refugee Women and Guidelines on Their Protection: An Assessment on Ten Years of Implementation. An independent assessment by the Women’s Commission for Refugee Women and Children. UNHCR Geneva [UNHCR women].

Van Hear N (2003) From durable solutions to transnational relations: home and exile among refugee diasporas. UNHCR/EPAU New Issues in Refugee Research. Working Paper No 83.

Women’s Commission for Refugee Women and Children (2002), Fending for themselves: Afghan Refugee Children and Adolescents working in Urban Pakistan. Mission to Pakistan -January 2002. www.womenscommission.org.

Society for Human Rights and Prisoners Aid (SHARP), Analytical Report on Minor Project for Children,), Islamabad, November 2003. www.sharp-pakistan.org

82 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

The Management of Nutrition in Major Emergencies, WHO 2000. Useful technical manual for understanding principles around nutrition in emergencies.

WFP Food and Nutrition Handbook.

The Sphere Project, Humanitarian Charter and Minimum Standards in Disaster Response 2000

UNICEF (2001) State of the World’s Children.

Scurvy outbreak and erosion of livelihoods masked by low wasting levels in drought affected Northern Afghanistan, Assefa F, Field Exchange, August 2001, Issue 13, www.ennonline.net

McPake B, Hongoro C (1995) Contracting out of clinical services in Zimbabwe. Soc.Sci. Med 41 (1) 13-24.

Refugee Health: An approach to emergency situations. Médecins Sans Frontières/TALC MacMillan, London 1997.

Cassels A (1995) Health sector reform: key issues in less developed countries. Journal of International Development 7 (3): 329-347.

World Bank (1993) World Development Report 1993: Investing in Health. New York, Oxford University Press.

World Bank (1999) Cambodia Public expenditure reviews. East Asia and Pacific Region. Washington DC: World Bank.

Jean, F. and Rufin, J.-C. (eds.) (1996) Economies des guerres civiles. Hachette. Paris 1996.

3. UNHRC Reports and Internal Assessments

Community Participation & Community Development approaches, Afghan Health Programme Pakistan UNHCR, Dr. Naveeda Rehman, Jan 2003, Descriptive, good quality.

Camp profiles for New Shamshatoo camp, Old Katcha Gahai camp,

Quetta UNHCR office description of Afghan refugee work in Balochistan provided by senior project officer Mr. Jusef Adam.

Assessment of performance and professional Integrity of the Project Directorate of Health (PDH), Dr. Ngogo, Health Advisor to UNHCR Peshawar office, June 2003. This is a very objective, frank and hard-hitting report that is concurrent with the findings of this evaluation in most respects. I found him correct concerning his assessment of the HQ of PDH in Peshawar, but somewhat hard on the health service (BHU) operatives, - but then, perhaps we were shown the better BHUs. However, this report is sloppily put together and therefore does not have credibility in some quarters. Numerical Rating of the various IPs generally concurrent with the findings of this evaluation

Strategy for Procurement of Medicines for Pakistan 2003, R.G. Elbro Sr. Regional Supply Officer, UNHCR, Islamabad. 21.1.2003.

83 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Numerical rating and comments on IP performance by UNHCR camp field officers and field clerks for Balochistan and NWFP.

Letter of appreciation to PDH NWFP, by SO Peshawar. 03/Pesh/Progr/102.

Consolidation of the PDH Structure. Proposals for the year 2004. Confidential report by UNHCR Public Health Officer, 27 October 2003.

UNHCR sub-office Peshawar. Health Sector Profile and Plans for the Year 2004. Public Health Officer UNHCR NWFP. 18 September 2003.

UNHCR Pakistan Update. UNHCR Islamabad, 31 October 2003.

UNHCR’s Disengagement Strategy. New Camps. UNHCR Islamabad 2003.

Pakistan Programme. UNHCR printout, unspecified.

UNHCR Representative’s Mission to Quetta, Balochistan. 17 to 21 August 2003.

PIVOT TABLE SUMMARY FOR PAKISTAN: Budgets by sector for all implementing partners/ implementing partner and by sector, 1994-2003.

Draft pharmacy management system guidelines. Dr. Naveeda Rehman, UNHCR Islamabad. 15.4.2003.

Voluntary Repatriation Operation Weekly Statistical Update. Prepared by Data processing. Statistical overview of Afghan refugees’ repatriation from Pakistan for period 07 November – 20 November 2003 compared to cumulative for 2003. Prepared by Data processing/GIS unit BO Islamabad.

UNHCR (2000) Handbook for Emergencies. Second Edition, UNHCR Geneva.

UNHCR Pakistan. Achieving our Objectives through Enhanced Partnerships. October 2003 (Matrix attempts to provide a general overview of the contributions made to the Pakistan refugee programme since September 2001 through strategic partnerships).

UNHCR (2001) Health, Food and Nutrition Toolkit, Tools and reference Materials to Manage and Evaluate Health, Food and Nutrition Programmes.

Agreement between the Government of Islamic Republic of Pakistan, The Transitional Islamic State of Afghanistan and the United Nations High Commissioner for Refugees governing the repatriation of Afghan citizens living in Pakistan, 2002.

Survey Report, Reproductive Health & EPI Coverage in Refugee Camps of Balochistan. December 2002. In collaboration with the technical assistance of WHO Quetta.

4. Implementing Partners, NGO Reports and Surveys, and Educational Documents

IRC Primary Health Care Programme in Hangu District, Dec 2003 report.

84 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Save the Children Fund (US), Participatory Rapid Appraisal, “A study on sustainable primary health care services” Afghan Refugee camps, Haripur and Ghazi, 8- 27.7.2003, Shaima Suleman Kheil et al. A good survey of SCF work in Haripur.

Danesh pre- & post-natal care awareness project, funded by UNICEF. A very good pictorial guide for TBA training in ANC, natal care and referral indications, - designed for illiterate persons.

Quarterly report referral management system in Quetta and Charman July –Sept 2003, Association of Medical Doctors of Asia.

Action Contre La Faim, Nutritional anthropometric survey, children between 6 and 59 months, new camps of Afghan refugees in Baluchistan province, Pakistan; March/April 2003.

Action Contre La Faim, Nutritional anthropometric survey, children between 6 and 59 months, new and old camps of Afghan refugees in NWFP, Pakistan, 2003.

Analytical report on Minor Project for children, Society for Human Rights and Prisoners aid, November 2003.

Balochistan provincial analysis of HIS report September 2003.

National Quarterly HIS report (July-Sept, 2003).

Reproductive Health in Refugee Situations, an inter agency field manual, 2000, UNFPA, WHO and UNHCR.

Association of Medical Doctors of Asia (AMDA), Quarterly progress report on Referral Management System in Quetta and Charman, July-September 2003.

Frontier Primary Health Care, Annual Report 2002.

Spatial and temporal patterns of anthroponotic cutaneous Leishmaniasis among Afghan refugees and local populations in NWFP, Baluchistan and Punjab, Pakistan, Dec 2002-March 2003, Health Net International and London School of Hygiene and Tropical Medicine.

Data of EPI vaccines and other medical stores in UNHCR/CCAR warehouse, Islamabad, for Afghan Refugee Health Care programme in NWFP, Baluchistan and Punjab. 1/12/2003.

AMDA referral summary for 2003.

“Still in need” Reproductive Health Care for Afghan refugees in Pakistan: Women’s’ Commission for Refugee Women and Children, 2003.

Reproductive Health during conflict and displacement, a guide for programme managers, RHR, WHO, 2000.

List of library books in the IRC BHU libraries in NWFP, per Dr. Tila Medical Director.

85 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Increasing Protection of Afghan refugees from Malaria and Leishmaniasis through Social Marketing of Insecticide Treated Mosquito Nets. Concept Paper for ITN Operations in 2004, HNI 2003.

5. Operational Partner Papers and Documents

WHO presentation on interactive assistance for Afghan refugees (in conjunction with UNHCR) in Balochistan province.

Household Food Economy Assessment, Refugee Camp Shamshatoo, NWFP, by Sustainable development policy Institute (SDPI), to the World Food Programme & UNHCR, 2003.

WFP/UNHCR Joint food needs assessment mission to Pakistan, 28.9.2003 – 10.10.2003.

National nutritional survey 2001-2002, Planning Commission Government of Pakistan, UNICEF, Pakistan.

WHO emergency response in the fallout of the Afghan crisis in Pakistan, WHO country office Pakistan, Islamabad April 2003.

Reproductive Health during conflict and displacement, a guide for programme managers, Dept of Reproductive Health and Research, WHO Geneva, 2000.

Concept Paper. Establishing Inter-agency Health Programme Evaluations in Humanitarian Crises. Inter-Agency Health Programme Evaluation. Concept Paper draft. 2003.

Nutrition Surveys of Afghan Refugees Living in Old Camps and Urban Host Communities, NWFP of Pakistan, April/May 2002, UNICEF, UNHCR and PD.

UNHCR/WFP Guidelines for Selective Feeding Programmes in Emergency Situations, February 1999 (even though there are clear guidelines on when to implement selective feeding programmes they were not being adhered to).

Memorandum of Understanding between UNHCR and WFP, July 2002 (Pilot country where WFP is responsible for final destination of food with IP’s which appears to be working well).

Report of Nutrition Survey in Existing Afghan Refugee Camps Balochistan, Pakistan December 2001, PDH, UNICEF, UNHCR, MCI.

Food and Nutrition Needs in Emergencies, guidelines jointly developed by UNHCR, UNICEF, WFP, WHO.

86

Annex 2: Chronology of Itinerary, Contacts and Activities

Itinerary

26 November 2003 arrival in Geneva (MM, MC) 167 27 November 2003 Port Moresby (PNG) – Cairns, Australia (GM) 28 November 2003 Cairns (Australia) – Hong Kong (GM) 30 November 2003 arrival in Islamabad (MM, MC, GM) 4 December 2003 Islamabad - Peshawar 7 December 2003 Peshawar – Kurram agency (FATA), via Hangu 9 December 2003 Kurram - Peshawar (via Kohat [MM], via Hangu [MC, GM]) 10 December 2003 Peshawar - Islamabad via Haripur 11 December 2003 Islamabad – Quetta 16 December 2003 Quetta - Islamabad 18 December 2003 Departure from Islamabad (MM, MC, GM) 18. December 2003 Dubai – Bangkok – Brisbane (Australia) (GM) 19 December 2003 Brisbane – Cairns (Australia) (GM) 19 December 2003 debriefing in Geneva (MM, MC) 20 December 2003 Cairns - Port Moresby (PNG) (GM) 20 December 2003 Departure from Geneva (MM, MC)

Contacts and Activities

Thursday 27 November 2003 UNHRC Geneva (MM, MC) Briefing with Dr Nadine Ezard, Senior Public Health Officer, HCDS (MM, MC) Briefing with Laurent Raguin, Senior Desk Officer CASWANAME (MM, MC) Briefing with Dr André Griekspoor, WHO/HAC and Nadine Ezard (MM, MC) Briefing with Fathia Abdallah, Senior Nutritionist, HCDS (MC) Telephone conference, chaired by Nadine Ezard, with Nepal evaluation team: Muireann Brennan (CDC), Oleg Bilukha (CDC), Marlène Bosmans (International Centre for Reproductive Health, Ghent, Belgium), Paul Spiegel, senior technical officer HIV/AIDS and Basia Tomaczyk (MM, MC) Briefing with Mark Rowland, Jan Kolaczinski and Kate Graham (HealthNet International/LSHTM)

Friday 28 November 2003 Briefing with Eddie Gedalof, Head of Health and Community Development Section (MM, MC)

Monday 1 December 2003 Collective briefing at UNHCR Islamabad (MM, MC, GM): Dr Naveeda Rehman, Health Co-ordinator UNHCR, Rita Richter, Programme Officer UNHCR, Naeem Durrani, Senior Programme Field Officer UNHCR, Tahiana Andriamamasomanana, Assistant Representative (Programme) UNHCR. Philip Karani, Acting Representative UNHCR, Dr Khaleef Bille, WHO Representative,

167 MM: Markus Michael, MC: Mary Corbett, GM: Glen Mola

87 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Jamshaid Qureshi, WHO National Consultant of Emergency preparedness, Dr Quaid Saeed Akhunzada WHO Emergency Medical Officer NWFP/FATA, Dr Shaida Fazel UNFPA Representative (partim)

Visit to slum populated by Afghan refugees in Islamabad I-11 (MM, MC, GM) Group interview with refugee men from Shamali and Parwan area

Tuesday 2 December 2003 Briefing by Col. Dr Altaf-ur-Rahman Khan, Medical co-ordinator Chief Commissariat for Afghan Refugees (CCAR), Afghan Refugee Islamabad Warehouse (MM, MC, GM)

Introduction to Deputy Secretary of SAFRON (Ministry of States and Frontier Region), Mr Noorullah and Dr Altaf (MM, MC, GM)

Visit to established urban Afghan refugee settlement in Islamabad Kachi-Abad, group interview with ‘village’ elders at the mosque (MM)

Meeting with Ms. Carrie Morrison of the Women’s Commission Reproductive Health project, Dr. Neelofar Zahid of American Refugee Committee and Dr. Maqsooda Kasi, Reproductive health advisor for Save the Children Fund Women’s Committee (MC, GM)

Wednesday 3 December 2003 Briefing by Imran Zeb Khan, Director Chief Commissariat for Afghan Refugees (MM) Briefing by Fahrunnisa Akbatur, UNHCR protection officer (MM)

Meeting with SAVEERA (A social services support & counselling services NGO funded by UNHCR), Director, Ms. Sharifa Panezai and two of her counsellors (GM)

Meeting with Polio Eradication office and National Institute of Health/Nutrition (need to get right names MC)

Introduction to Gen (R) Aslam, Director General of Health, MoH. In presence of Col. Dr Altaf-ur-Rahman Khan (MC, MM)

Security briefing at UNDP by Mr Khawar Nadeem (MC, MM, GM)

Thursday 4 December 2003 Briefing at UNHCR Sub-office Peshawar, by Mr Niaz Ahmad, acting Head of Office, Ahmed Warsame, Senior Program Officer, Maricela Daniel, Senior Programme Officer, Munazza Hadi, Saima Imtiaz, Dr. Sasha Ali (MC, MM, GM)

Courtesy Visit to Commissioner for Afghan Refugees, Brig. (R) Ahmad Alizai and staff (MC, MM, GM)

Briefing by Dr Jahweed Parveez, Director, Project Directorate Health (PDH) and staff (MC, MM, GM)

88 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Interview with Dr Jahweed Parveez, Director PDH (MM) Interview with Dr Saima Abid, Referral Medical Officer PDH (MM) Interview with Mr Bakut Karam, Pharmacist PDH (MM) Interview with Bushra, Dr. J. Afrdidi, Mr Kunshid, PDH employees’ association (MM) Interview with Dr. Tehsin Fatima, Deputy Director for Reproductive Health PDH (GM) Interview with Deputy Director for PHC, Dr. Muhammad Saleem PDH (GM) Interview with Dr. Ishaq, IRC/CDC funded Director of the HIS coordination unit PDH (GM)

Interview with Deputy Director for PHC, Dr Muhameed Saleem, PDH (MC) Interview with Dr Jawad Habib Khan, Deputy Director for EPI PDH, (MC) Interview with Dr. Tehsin Fatima, Deputy Director for Reproductive Health PDH (MC)

Visit to Katcha Gorhi refugee camp in suburban Peshawar (MM, GM) Group interview with young men and children (MM) Individual interviews with four young fathers about reproductive health issues (GM)

Visit to Kuwait Joint Relief Committee Hospital, meeting with director Abdul Salam Al-Sharif, (MC)

Friday 5 December 2003 Group Interview with five UNHCR Field Officers (MM, GM)

Meeting with Salim Akhdar, and three other staff members from WFP. Update on food pipeline and ration size (MC)

Group interview with representatives of Implementing Partners: 1) Union Aid for Afghan Refugees, 2) World Assembly for Muslim Youth, 3) International Rescue Committee, 4) Church World Service for Afghan Refugees, 5) Women’s Commission for Refugee Women and Children 6) Pakistani Red Crescent Society 7) Afghan Medical and Welfare Society 8) Save the Children US, 9) HealthNet International, 10) Association for Community Development, 11) ILO, 12) International Medical Corps, 13) Frontier Primary Health Care, 14) Kuwait Joint Relief Committee (MM, GM)

Workshop with representatives of Implementing Partners: Union Aid for Afghan Refugees, Women’s Commission for Refugee Women and Children, Save the Children US, HealthNet International, Association for Community Development, International Medical Corps, Frontier Primary Health Care. (MM,GM, MC [partim])

Visit to Mercy referral hospital, visited wards and met with authorities (MC)

Briefing by and interview with Masti Notz, Head of UNHCR Sub-office Peshawar, and Ahmed Warsame, Senior Programme Officer (MC, GM, MM)

89 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Saturday 6 December 2003 Interview with Dr. Jahil ur Rahman Directorate of Health NFPF, and Deputy directors for MCH (Dr. Shabina), EPI (Dr. Waheed), and Malaria control (GM, MC, MM)

Visit to Shamshatoo camp (GM, MC, MM): BHU evaluation, interviews with staff (GM) Exit interviews with refugee women clients of BHU (GM) Focus group with male Community Health Workers in Shamshatoo camp (MM) Interview with village elder in Shamshatoo camp (MM) Visit to IMC BHU, interview with staff and exit interview with patient (MC) Interview in home with mother and two children (MC) Group interview with young refugee family fathers in urban slum in Yusufabad, Peshawar (MM)

Interview with Mr Osama, UNICEF representative Peshawar (MC, GM)

Sunday 7 December 2003 Travel to Kurram Agency, accompanied by Dr Naveeda Rehman, UNHCR and Dr Quaid Saeed Akhunzada, WHO (MC, GM, MM) Stop in Hangu, Interview with Dr Tila Khan, project Manager IRC (MC, GM, MM) Visit Emergency Obstetric Centre (MC, GM) Interview with UNHCR field officer Mr Bashir (MM) Interview with Dr Quaid Saeed Akhunzada, WHO (MC)

Monday 8 December 2003 Visit to refugee village Muzafar Kot (MM) Visit to BHU, individual interviews with Medical Officer, Lady Health Visitors and dispenser; one exit interview with mother of two children patients. Group interview with village elders Interview with male family members of one randomly chosen refugee family

Visit to Visit to Sateen old camp (GM) Evaluation of BHU Group interview with six refugee women and LHVs (partim)

Visit to Sada Civil District hospital (GM) Interview with FSMO for PDH Dr. Amat Khan Marwat (GM)

Visit to Asgaro camp (MC) Visited BHU run by PDH and interviewed staff (MC) Interviewed one patient waiting to be seen at clinic (MC) Visited and interviewed three households and interviewed mothers with young children (MC) Visited Inter SOS women’s skills centre (MC) Interview with Dr Abdul Rahman, Chief Medical Officer WAMI (MM) Interview with Dr M. Ajmal, Medical Director Shasho Hospital (MM)

Interview with Dr Amal Khan Marwat, PDH field Supervising Medical Officer, Kurram Agency (MC)

90 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Tuesday 9 December 2003 Visit to Kohat District (MM) Visit to Hangu district, Shapur BHU, interview with Dr Iftehar, Medical officer Interview with PDH FSMO of Kohat province, Dr Javaid Afridi (MM) Household interview with two randomly selected refugee women (MM) Interview with 15 male community health workers about RH issues at Shapur BHU, (GM) Visit to IRC supported camp called Thall, Hangu district (GM, MC) Visit to BHU (GM) Meeting with group of female health workers (MC) Visited and interviewed two families in their homes (MC) Arrived back in Peshawar at 7pm

Wednesday 10 December 2003 Visit to Haripur district, introduction by medical director Dr. Pennez Shankat and staff of Save the Children US (MM, MC, GM). Visit to refugee camps Focus group with male Community Health workers (MM) Interview with Dr Binash, Medical Officer BHU (MM) Interview with randomly selected male family father (MM) Interview with Farkhanda Anwar, UNHCR Field Officer (MM) Evaluation of Deenda BHU (GM) Group interview with 13 refugee men (GM) Group interview with six refugee women (GM) Interviewed SCF US LHW Fahima (MC) Randomly selected 3 households and interviewed women (MC)

Thursday 11 December 2003 Welcome by Yusef Ahdam, senior programme manager and acting head of sub-office UNHCR Quetta (MM, MC, GM) Introduction by Commissioner for Afghan Refugees of Baluchistan, Brig (R) Rajah Mumtaz, in presence of Dr Tahira, WHO and Dr Salim Akbar, PDH (MM, MC, GM) Group interview with representatives of Operating Partners, Nadir Gul Barech, UNFPA; Mahboob Ahmad Bajwa, UNICEF; Adham Effendi WFP; D. Tahina Kamal, WHO (MM, MC, GM) Group interview with UNHCR Quetta Field and Protection Officers (MM, MC, GM)

Friday 12 December 2003 Interview with Yusef Ahdam, senior programme manager and acting head of sub- office UNHCR Quetta (MM) Visit to tertiary hospital BMC (Bulan Medical Complex) and evaluation of AMDA referral system (GM)

Workshop with representatives of Implementing Partners, SCF US, MSF Holland, DANESH, AMDA, ARC (MM, MC, GM) Meeting with Dr SM Salem Akbar, Project director Health (MC) Meeting with Javed Iqbul Malik, HealthNet International (MC)

Saturday 13 December 2003 Visit to Latifibad Camp (MC) Visit to AMDA run BHU with interview of staff including Dr Shanawaz (MC)

91 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Interviewed group of men, approx fifteen, including two health workers (MC)

Visit to Mohammed Khel refugee camp (MC, GM) Group interview with 25 girls aged 10-14, part of a peer group education group (GM) Group interview with 20 Female Health Workers/TBAs (GM) Visit to BHU operated by PDH, group interview with 30 male CHWs (GM) Visit to BHU operated by PRCS with interview of staff (MC)

Visit to BHU of Pakistani Red Crescent Society, interview with Dr Ahmer, Provincial Health Officer PRCS (MM) Visit to MoH BHU in Kili Kotwal, interview with MO Dr Saleh (MM) Visit to Aryana Development Foundation BHU in Moghulabad Eastern Bypass, interview with ADF director Sakhi Masraf, SDF health coordinator Dr Naseem, and Dr Farrukh Rahum Hasni, Deputy Health Programme Manager Mercy Corps (MM) Group interview with male refugees and children in urban slum in Moghulabad Eastern Bypass (MM) Visit to urban refugee neighbourhood Akhrot Abad (MM)

Sunday 14 December 2003 Team discussion (MM, MC, GM)

Monday 15 December 2003 Visit to Chaman (MM, GM, MC) Visit to Chaman district hospital, interview with Medical Superintendent Dr. Wali Muhammad Babai (GM) Exit interview with widow at AMDA BHU in Dara II camp (MM) Interview with Plan Pakistan ambulance drivers (MM) Visited AMDA operated BHU and interviewed MCH and EPI staff (MC) Interviewed some mothers and children in waiting area of BHU (MC)

Interview with Dr Judy McConner, project manager MSF Holland Landi Karachi camp Visit to BHU run by MFS Holland (GM) Interview with Health worker Soni Mohamed, MSF Holland, (MC)

De-briefing with Yusef Ahdam, senior programme manager and acting head of sub- office UNHCR Quetta (MM, MC, GM)

Tuesday 16 December 2003 Feedback session in Quetta with UNHCR staff and representatives of OPs (WFP, UNICEF, WHO), IPs (AMDA, SCF US, ARC), other NGOs (MSF H, PRCS, Mercy Corps, DANESH, Afghan Teachers’ Organisation) and PDH/CAR (GM)

Wednesday 17 December 2003 Feedback session in Islamabad with UNHCR staff, donor representatives (ECHO, JICA), SAFRON/CCAR/PDH, OPs (UNFPA, WFP), IPs (Frontier Primary Health Care, AMDA, SCF US, HealthNet International, WAMY, IMC, IRC) and other (Union Aid for Afghan Refugees PRCS, CWS, Women’s Commission) (MM, MC, GM) Feedback session with UNHCR staff, Tahiana Andriamasomanana, Assistant Representative (Programme), Naeem Jan Durrani, Field/Programme Officer (MM, MC, GM)

92 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Thursday 18 December 2003 Travel from Islamabad

Friday 19 December 2003 De-briefing session at UNHCR with Dr Nadine Ezard, Senior Public Health Officer, HCDS, Laurent Raguin, Senior Desk Officer CASWANAME, Fathia Abdallah, Senior Nutritionist HCDS and other UNHCR staff, Dr André Griekspoor, WHO/HAC, and a representative from the US Mission/BPRM (MM, MC) De-briefing session with Cesar Dubon, Head of Desk CASWANAME (MM, MC)

93

Annex 3: Terms of Reference

JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES

FOR AFGHAN REFUGEES LIVING IN PAKISTAN

Following the discussion on the concept paper ‘Establishing Inter-agency Health Programme Evaluations in Humanitarian Crises’, a number of contexts were discussed where the first evaluations could be implemented. Afghan refugee populations living in Pakistan have been identified for the second pilot, following a joint evaluation of services provided to Bhutanese refugees in Nepal. Further studies are planned in Zambia, DRC and Burundi.

This joint evaluation is not an evaluation of the programme(s) of any single agency. Together with all relevant stakeholders, including the affected population, it aims to look at the overall health status, risks to health and the range of services available to the refugee population to achieve and maintain an acceptable health status. The emphasis for the assessment of the performance and effectiveness of these services will be on the health and nutrition sectors. A short description of the Afghan refugee health programme in Pakistan can be found below. The country team has developed a series of more specific questions that are elaborated under the specific evaluation criteria.

Developed jointly with WHO, UNHCR will take the lead in organizing this evaluation and facilitating the logistics and other support needed in country. Our counterparts in the programme area will define the relevant health agencies concerned with health service provision to the affected population. These will include the agencies involved in the health sector coordination, but when indicated, others will be invited to join, including relevant donors. Together, they will fine-tune the purpose and objectives for the joint evaluation.

The evaluation will be undertaken in a manner that is consistent with UNHCR evaluation policy.

The evaluation will dovetail with the Interagency Working Group on Reproductive Health's Global Reproductive Health Evaluation. The evaluation team will be joined by a Pakistan-based representative of the Women's Commission, a member of the Global Reproductive Health Evaluation standing committee, building on their recent reproductive health evaluation of refugee camps (August 2002 - June 2003).

In addition, the evaluation mission follows one month after the completion of the Joint Food Assessment Mission (JFAM) conducted jointly by UNHCR and WFP. This mission will follow-up on specific health and nutrition recommendations from that mission including on selective feeding programmes. It also provides an opportunity to make observations on linkages between the different processes, both the health services evaluation and the JFAM.

95 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

BACKGROUND: The Afghan Refugee Health Programme in Pakistan There are some 1.5 million Afghan refugees of concern to UNHCR living in Pakistan, UNHCR’s largest caseload (See Map).

Map: Afghan refugees in Pakistan

The majority of the refugees live in long term ‘refugee villages’, (from where approximately 1.5 million were repatriated last year), although a new influx of some 200,000 refugees last year has been accommodated in ‘new camps’ in Balochistan and North-West Frontier Province (NWFP) The majority of refugees (1.2 million) live in northern NWFP, with approximately 300,000 in Balochistan and 40,000 in Punjab (Table 1).

96 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Table 1: Registered refugee population per province

Province Population % No of health units NWFP 1,204,862 77.4% 83 Balochistan 310,736 19.9% 31 Punjab 40,000 2.57% 04 Total 1,555,980 99.87% 118 Health services are provided collaboratively between UNHCR, some 14 health- implementing partners, and the Government of Pakistan, through 118 health units composed of 104 BHUs, two Central Health Units; ten Sub-Health Units; and two Mobile Health Units (see Table 2).

Table 2: Health Unit status by Implementing Partner, Afghan Refugee Health Programme, Pakistan

Implementing/operational Partner Province Population Total # Health Units Government of Pakistan (GoP) Project Directorate Health (PDH) NWFP NWFP 490,609 35 PDH-Balochistan Balochistan 142,320 15 Commissioner for Afghan Refugees Punjab 40,000 3 (CAR) Union Aid for Afghan Refugees (UAAR) NWFP 163,120 9 Kuwait Joint Relief Committee (KJRC) NWFP 104,503 8 International rescue committee (IRC) NWFP 122,163 9 Save the Children (SC-US) NWFP 109,000 7 Church World Services (CWS) NWFP 54,490 3 Humanitarian Medical and Relief Body NWFP 34,000 1 (HMRB) Afghan Medical & Welfare Association NWFP 45,000 2 (AMWA) Frontier Primary Health Care (FPHC) NWFP 32,883 3 World Assembly of Muslim Youth (WAMY) NWFP 7,184 1 International Medical Corp (IMC) NWFP 41,901 4 American Refugee Council (ARC) Balochistan 97,870 10 Asian Medical Doctors Association (AMDA) Balochistan 37,885 3 Pakistan Red Crescent Society (PRCS) Balochistan 9,000 2 Medicines Sans Frontier (MSF-Holland) Balochistan 34,461 3 1,566,389 118 Each BHU provides outpatient primary health care services to a population of approximately 10,000 people. BHUs are linked to a network of male and female community health workers (operating in the refugee villages) and refer to the Ministry of Health (MoH) for in-patient and other second and third level health services. The health programme focuses on women and children’s health in the overall framework of a primary and reproductive health care programme. See Table 3 for key health and nutrition indicators in 2002.

97 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Table 3: Health and Nutrition Indicators in 2002:

Key Indicators Provinces NWFP Balochistan Punjab Measles vaccination coverage 80.0% 75.4% EPI coverage 76.7% 50.8% 90% Crude Mortality Rate 0.23/1000/mth 0.4/1000/mth 0.2/1000/mth Under-five Mortality Rate 0.48/1000/mth 0.5/1000/mth 0.31/1000/mth Under-one Mortality Rate 26.85/1000/mth 36/1000/mth 30/1000/mth Under-five malnutrition rate: Global acute malnutrition (in old 4.7% 6% 2.7% camps) Severe acute malnutrition (in old 0.7% 1.7% 0.4% camps) Global acute malnutrition (new 8.1% 5.8% - camps) Severe acute malnutrition (new camps 1.6% 2,8% - Water/litre/person 15 15 15 Breast feeding practices Breast feeding at birth 99.1% 98% 99.6% Breast feeding at two years-old 62% 78% 59% Weaning practices Starts weaning at 3-6 months-old 61.8% 70.2% 81% Starts weaning at 6-9 months-old 16.2% 13% 16% Antenatal coverage (three visits) 51% 46% 79% Contraceptive Prevalence Rate 3% 5.8% 7%

Key elements of the health programme include

Children less than one year of age and women of reproductive age (14-49 years) are included in the routine vaccination programme. Children are immunized against six communicable diseases and women against tetanus. Vaccination coverage, however, is inadequate (Table 3) with ≤ 80% children vaccinated against measles. See survey reports (2002) for more details (Annex).

Although the reported malnutrition rate is less than 10%, a supplementary feeding programme was opened in September 2003 (Table 4). There are some concerns about coverage, impact and appropriateness of the programmes (see Joint Food Assessment Mission report, October 2003).

Table 4: Supplementary Feeding Programme Centers in Balochistan and NWFP 2003

Provinces No of Population Health SFP Referral to TFP camps units Balochistan 6 123,000 8 6 Organized NWFP 7 73,000 7 6 Organized in all camps except in new Shamshatoo UNHCR provides funds for procurement of essential drugs, consumable and laboratory supplies through partners. There are reportedly concerns from some refugees regarding quality, drug stock outs, and access, especially in Provincial Directorate of Health supported BHUs in Balochistan (see Mission report, Njogu

98 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Patterson, August 2003). A visit by headquarters supply officer was conducted in September 2003.

Referral systems to in-patient care units or for secondary and tertiary care are reportedly expensive (particularly in Balochistan, where the system is supported by the NGO AMDA) and erratic (particularly in NWFP where there is no uniform policy for referral).

Reproductive Health (RH) services, although established at the outset, enjoy low coverage (see also Still in Need, Reproductive Health Service for Afghan Refugees in Pakistan, Women’s Commission for Refugee Women and Children, October 2003). All components of the Minimum Initial Service Package (MISP) are in principle in place, as well as safe motherhood, family planning, prevention and treatment of sexually transmitted diseases and awareness on HIV/AIDS. Survey results (2002) from Balochistan new camps suggest that 37.40% of pregnant women had at least one antenatal visit, 27.51% two antenatal visits and 18.78% completed three visits. At least 13.6% of the married women had used a family planning method and 6.7% are currently using contraceptives. 93.5% women in new camps preferred to deliver child at home. 78.0% women were assisted during delivery by relative. A health worker (doctor, Lady Health Visitor or a trained health worker) assisted in 19% of the deliveries. Survey results from NWFP (2002) suggest that 21.23% of pregnant women had three or more antenatal visits. 72.1% women knew at least one family planning method, 16.5% had used contraceptives and current contraceptive prevalence was 7.9%. The most popular method of contraception was Depo-Provera, accounting for 65%, followed by the oral contraceptive pill at 15.8%. See Annex for survey report.

A Health Information System (HIS) has been established for key morbidity and mortality data, but systematic and rigorous data is not yet widely available. A purpose-built software package has been developed by CDC-Atlanta, revised in 2003. HIS core staff training has been conducted, and district staff training is underway.

HIV/AIDS services include: Health education and awareness for health workers and community. Universal precautions at health units including safe disposal of waste material, syringes and needles and ensure sterilization of equipment, etc. Training of health staff and health workers on HIV/AIDS. An NGO (AMAL) is engaged in training of Master Trainers on HIV/AIDS for Punjab project staff. Twenty-eight staff from different sectors including health, education, community services and general administration will be trained. They will be responsible to design training for other staff and health workers. Training of staff and health workers in other provinces is a continuous process. Production of HIV/AIDS health education material including flip charts and health guide in appropriate local languages. The international NGO ARC in Quetta is establishing voluntary Testing Centre (VTC) in Mohammed Khel in coordination with UNAIDS.

99 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Promotion of condom use is in under discussion. Condom utilisation is estimated at less than 1% at present. There is a need to involve male health workers for condom promotion. Water supply of 15 litres per person per day is at the internationally accepted minimum for emergency situations, even in long standing camps.

PURPOSE AND OBJECTIVES:

The main purpose of the joint evaluation in Pakistan is to promote the sharing of lessons learned and good practices among implementing agencies, ministries of health, UN agencies, donors and other relevant stakeholders.

Specific objectives include:

To promote quality in the programmes and services implemented by humanitarian agencies and National Health Service providers. To promote application of appropriate standards in health programmes and services provided. To promote coordination and collaboration among implementing agencies and relevant stakeholders, including MoH and affected populations To assist field actors to plan for more consistency between refugee health programmes and national health programmes, to facilitate integration when this would be appropriate. To prioritize the health services required and review the allocation of available health services against these priorities To promote programme re-orientation to ensure that the health and nutrition needs of refugees continue to be met in the context of scaling down of support to refugees living in Pakistan. To optimize learning, the evaluation will not only describe the health status and level of performance of the health system to answer the questions of did we do the right things in the right way, but it will also look into the underlying question of why things are the way they are.

USE:

UNHCR is the primary user of this joint evaluation. They will coordinate the action on the findings and compliance for approved evaluation recommendations. Other key users include the UN agencies and NGOs involved in the delivery of health programmes, and the Pakistan DHS. Furthermore, the evaluation will inform donors on achievements made and the need for future support. Recommendations from the evaluation will assist in programme reorientation in the context of scaling down of external support to refugee operations in Pakistan.

Results from the evaluation will inform the development of a coherent multi-agency assessment and evaluation framework for complex emergencies and refugee situations, linked to the Good Donorship initiative.

100 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

SCOPE:

As mentioned earlier, this joint evaluation is not an evaluation of the programme(s) of any single agency. Together with all relevant stakeholders, including the affected population, it aims to look at the overall health status, risks to health and the range of services available to the refugee population in order to achieve and maintain an acceptable level of health. The emphasis for the assessment of the performance of services will be on health and nutrition programmes.

In particular, it will look at the following evaluation criteria:

What is the overall impact of the humanitarian interventions on the health status (and protection) of the refugees? Are the health programmes relevant to the health needs of the refugees and are they effectively addressing these needs? Are the health programmes adequately coordinated between the relevant international and national stakeholders? Are the health programmes appropriate from the perspective of the refugees? Are the programmes coherent with UNHCR and national health policies? Connectedness: Where and when indicated, do health programmes take into account a possible integration within the Pakistani health system? Is there adequate coverage of the refugee population? Efficiency: what is the overall budget for the health programmes and is this adequate and/or adequately used to address humanitarian needs? Evaluation team:

Agencies involved in the design of the concept paper have been approached and requested to nominate staff members for this evaluation. Through these contacts, two specialists with complementary competencies have been identified i.e. Public Health expert (PH) and a nutritionist (Nut). A Reproductive Health specialist (RH) from a partner NGO based in Pakistan will join the team. The team will appoint one team member as team leader. Together, the team will decide on division of tasks. To give an indication for the respective areas to be covered by the different team members, the key questions have indications as to who would take the lead on these questions.

Methodology:

Upon arrival, the team will establish contact with relevant stakeholders, including UNHCR, WHO, WFP, MoH, UNFPA, UNICEF, Red Cross, donors including the US government, ECHO, and DFID Within approximately three days, the team will fine- tune the TOR and make a work plan and time schedule for the field visits, which should include long-term settlements and new camps.

A range of assessment and evaluation tools will be provided to the team for use as appropriate. Comments on the experience of using these tools should be provided in the final report, but formal evaluation of these tools does not form part of the terms of reference.

101 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

The evaluators and the stakeholders will develop a detailed methodology to be used for assessing the programme and it is expected that these will include:

Interviews with key-stakeholders, including a cross section of the affected population. Focus group discussions with refugees Review of health data and the methodology through which they were collected (including assessing whether methodology developed for the SMART initiative was or can be applied). Review of available reports to identify trends and health status of the refugees. Review of community-based surveys and the appropriateness of the methodology used. Review of Pakistan health care policies and guidelines, and compare them to the Afghan refugee health programmes. Observe health services as they are provided and conduct exit interviews with care recipients. Review findings of other evaluations conducted, including JFAM. When necessary and feasible, community based survey to fill information gaps. Review, or develop, with key stakeholders a ‘package’ of indicators that can be used to monitor performance of the health sector and other key factors that influence health. Identifying lessons, reporting and debriefing

To the extent possible, the evaluation will be participatory, interactive and a learning process for key service providers in the health sector. At the conclusion of the exercise, the evaluators are expected to:

Conduct one/two day debriefing workshop with the stakeholders in one or two locations. Preliminary findings will be shared with all partners. The evaluators will assist the agencies to draw a tentative plan of action based on the recommendations, including possible technical follow up to address identified problems. Conduct a two-to-three hour debriefing in Geneva with UNHCR, WHO and donor agencies that may be interested. They will highlight the main findings, recommendations and the tentative plan of action agreed with the key stakeholder in Pakistan. Submit a written report with sub-sections on the methodology of assessment (including comments on tools used), findings and recommendations, plus lessons learned on the process of joint health evaluation. Length will be 40-60 pages, with an executive summary of no more than three pages. Further guidance for the report can be found in the ALNAP ‘quality proforma’. Planning

Create a time schedule for the preparation, implementation, report writing and debriefing. The maximum number of days for each phase should be indicated as well as deadline for the final report.

102 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

KEY QUESTIONS

Impact: (PH, RH, Nut)

Did the overall humanitarian intervention achieve acceptable health status and protection of the refugee population? Can a contribution to the changes in the health status of the target population be attributed to the humanitarian interventions? What proportion to national and/or community effort? Are there any unforeseen negative impacts? Assessment questions:

What is the baseline and trend demographic data concerning the refugee population?

What is the health and nutrition status, what have been the trends?

How do indicators compare to international and national benchmarks?

Assess the health information reporting system:

Review data collection, submission, collation, analysis and dissemination of monthly and annual reports, including mortality and malnutrition data collection systems. To the extent possible, assess whether HIS data captures a significant proportion of the total number of deaths among the general population groups and among children aged five and under. Assess whether HIS is used to direct planning and resources allocation to meet the needs of the beneficiaries. Assess the feasibility and appropriateness of the use of the new HIS software developed collaboratively with CDC, and the extent to which it is being used. Relevance: (PH, RH, Nut)

Are existing vulnerabilities, health risks and determinants of health adequately identified? Do the humanitarian interventions address the most important needs? Are those that contribute most to the main health problems adequately prioritised and addressed through the humanitarian interventions and the range of services in the PHC programmes? Were appropriate and timely adaptations made in response to the any changes in the context? Assessment questions:

Assess main vulnerabilities, health risks and determinants in relation to the humanitarian interventions. What are strategic priorities and how were these determined?

103 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Effectiveness: (PH, RH, Nut)

How do interventions and programmes compare against quality standards, as defined in national and/or international guidelines (WHO, Sphere, etc.)? Were there any unforeseen/foreseen negative side effects, any unforeseen positive side effects? Why, or why not? Assessment questions:

Assess the quality of PHC services and food security (health, nutrition, water and sanitation, shelter, food). Document the strengths, the weaknesses, and gaps and propose corrective measures where possible. In particular, the control and treatment of malaria and Leishmaniasis should be mentioned. Recommendations for documenting ‘best practices’ or innovative approaches to disease control should be made. As international references for range of possible services and standards for performance, the Sphere handbook will be used and/or the UNHCR indicators where appropriate. To the extent possible, compare the refugee PHC programme to the national PHC programmes. Take into account the characteristics of the context; protracted, emergency, repatriation, consistency/differences among service providers (i.e. NGO or NGO/MoH) and infrastructure and level of care provided, staffing, skills, use of international, national and refugee staff and gender balance etc.

Assess the nutrition status (Nut)

- Review community based nutrition survey, its quality and follow up action, and where appropriate advise. - Establish whether there is any system in place for detection of micronutrient deficiency. - Assess the prevalence of diseases associated with micronutrient deficiency: anaemia, malaria, intestinal infestation, measles, and any measures taken to address them, etc and propose interventions where feasible. - Is there regular distribution for Vitamin A supplementation? If so, what is the coverage rate? - Review selective feeding programme, needs, coverage and adequacy of interventions, and provide advice for improvement. - Examine food distribution monitoring system in place that ensures adequate supply and fair distribution. (e.g. food basket surveys, post distribution monitoring etc) Assess the referral system to secondary and tertiary health institutions (PH, RH, and Nutrition)

Access to secondary and tertiary institutions. Availability and the use of referral guidelines. Transportation of critically ill cases. Review critical incidents such as patients who suffer severe complications with a view of identifying gaps and weaknesses in the referral system.

104 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Review the capacity of secondary/tertiary institutions to cope with/accept referrals. Assess MCH/FP including RH services (RH):

Staffing patterns and training. Coverage of services: CPR, ANC and PNC coverage, TT coverage, proportion delivered by trained health workers, etc. Determine childhood vaccination coverage using the existing database and other channels such as focus group discussions to gain insight about factors that may hinder EPI uptake. Identify to the extent possible factors that hinder provision of RH. Review critical incidents such as maternal death with a view of identifying gaps and instituting corrective measures. Review existing STI/HIV/AIDS prevention, promotion, and care and treatments programmes and propose interventions where appropriate using the IASC matrix as a reference point. Coordination: (PH, RH, Nut)

Are the interventions adequately coordinated between all relevant stakeholders? What is the role of the national health authorities in coordination Are there shared and/or joint assessments, planning monitoring and evaluations done? Is there adequate coordination between sectors, and development organizations e.g. in HIV/AIDS Programmes? Is there an attempt at cross-border coordination? Assessment questions:

Are there overviews of who does what where? What are coordination mechanisms? Special task forces to deal with specific issues? Evidence for joint strategic programme monitoring and evaluation? How is TB treatment coordinated with the national programme in Pakistan and in Afghanistan (for returnees)? Is there coordination of malaria and Leishmaniasis prevention and treatment nationally and with Afghanistan?

Coverage: (PH, RH, Nut)

To what extent do the interventions reach the specific target population? To what extent did refugees have access to project services? Was anyone excluded from services or are there any differences in access within the refugee population? Assessment questions:

Assess whether the PHC services are geographically, socio-culturally accessible, and whether they are affordable. Identify barriers that hinder their use. Does host population have access to refugee services?

105 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Connectedness: (PH, RH, Nut)

Do humanitarian interventions consider longer-term issues? Integration efforts in settlements and in camps: involvement of District Health Management Teams, potential for hand-over to these Teams in the long run Is there adequate attention to reducing vulnerabilities and strengthen local capacities? Are health and nutrition policies and guidelines in line with national health policies and guidelines? Are international and national NGO interventions conducted supportive to national health system or are they conducted in parallel? What should be desirable levels of performance and standards for services adjusted to the context that would allow sustaining these services? Assess the linkages of the refugee programme to national institutions, collaboration with national institutions for the control of diseases in the camps and in refugee hosting areas. Are refugee health workers participating in national continuing education?

Coherence: (PH, RH, Nut)

Is there an overall humanitarian strategy and/or a health sector specific strategy? Do humanitarian agencies use it to guide their interventions and ensure that they are complementary? Appropriateness and Participation: (PH & RH)

Are the humanitarian interventions appropriate in the perspective of the affected population? (culturally, socially, and addressing their demands) To what degree does the affected population participate in the humanitarian interventions and decision-making? Assess the community participation and involvement in health care services:

- Assess to what extent the community is involved in identifying health priorities, proposing interventions and their involvement in health care delivery. - Number of trained Community Health Workers (CHW) by gender, by age (youth and adolescents, middle aged and elderly) and the services they provide. - Assess the training needs of CHWs in relation to their responsibilities. Are they adequately trained to carry out their work? Does the curriculum cover RH, HIV/AIDS/ condoms, and endemic diseases and community mobilisation? - Assess the relationship between community health workers working with refugees and national programmes, particularly the Lady Health Visitor programme. Efficiency: (PH)

What is the total budget, including UNHCR and other contributions? Are there opportunities for improving the results within these available budgets?

106 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

How do per capita budgets compare to national and international indicators? Assess the efficiency of the referral system.

107

Annexae

Action Contre le Faim (ACF), Nutritional anthropometric survey reports, Balochistan and NWFP Pakistan, 2003. Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP), Quality Proforma, May 2003. High Commissioner UNHCR, Mission Report, August 2003. Rehman, UNHCR, Afghan refugee Health Programme in Pakistan, September 2003, (unpublished) UNHCR Reproductive Health and EPI Coverage in Refugee Camps of Balochistan, November 2002. UNHCR Reproductive health and EPI coverage survey in refugee camps in NWFP, November 2002. UNHCR/WFP, Memorandum Of Understanding between the Office of The United Nations High Commissioner For Refugees (UNHCR) and the World Food Programme (WFP), July 2002. UNHCR/WFP Guidelines for Selective Feeding Programmes in Emergency Situations, Feb 1999. WHO/UNHCR Inter-Agency Health Programme Evaluations in Humanitarian Crises, Concept Paper, 2002. Women’s commission for refugee women and children. Still in need: reproductive health care for Afghan refugees in Pakistan. Oct 2003 http://www.womenscommission.org/pdf/Pk_RH.pdf.

Other Suggested Background Materials:

Bartlett L, et al. Maternal mortality among Afghan refugees in Pakistan, 1999-2000, The Lancet; Feb 23, 2002, Vol. 359: 643-649. Hynes M et al. Reproductive health indicators and outcomes among refugee and internally displaced persons in post emergency phase camps, JAMA; Aug 7, 2002, Vol. 288: 595- 603. WFP/UNHCR. Draft Joint Food Assessment Mission Report. November 2003 (unpublished). Brennan et. Al. for UNHCR. Report of health assessment mission to Bhutanese refugee camps in Bangladesh. November 2003 (Unpublished). Reproductive Health for Refugees Consortium. Monitoring and Evaluation Tool Kit (draft for field testing). http://www.rhrc.org. The Sphere Project, Humanitarian Charter and Minimum Standards in Disaster Reponses (second edition). Geneva, 2004. UNHCR, Health Toolkit, Geneva 2001. Médecins sans Frontières (MSF), Refugee Health: An approach to emergency situations, 1996. Palmer CA, Lush L, Zwi AB. The emerging international policy agenda for reproductive health services in conflict settings. Social Science and Medicine 49 (1999) 1689-1703.

109

Annex 4: List of Implementing Partners for Health

Partner Status Location UNHCR Cofunding Camp No. of No of Budget population Health Units staff 2003, US $ served PDH GoP NWFP 2,131,969 No 490’609 36 746 PDH GoP Balochistan 607,722 No 142’320 15 237 CAR GoP Punjab 186,285 No 40’000 3 (2 BHUs 70 1 CHU) CCAR GoP Islamabad 29,606 Yes All Afghan Management 12 HCR refuges funded International INGO NWFP 272,631 Yes 122.163 9 140 Rescue Committee Kuwait Joint INGO NWFP 250,558 No 104.503 8 123 Relief Committee Union Aid for AFG NWFP 226,836 No 163,120 9 129 Afghan NGO Refugees Health Net INGO NWFP 203,665 Yes (disease - 34 International specific) Association for Nat. NWFP 160.304 No (disease - 22 Community NGO specific) Development Pakistan Red Nat. NWFP 164,889 No (Referral) - 26 Crescent NGO Society Afghan AFG NWFP 54,469 Yes 45,000 2 24 Medical NGO Welfare Association Save the INGO NWFP 247,236 Yes 109,000 7 160 Children – US Frontier Nat. NWFP 81,979 No 32,883 3 39 Primary Health NGO Care Church World INGO NWFP 73,726 Yes 54,490 3 36 Service World INGO NWFP 47,516 No 5,184 1 20 Assembly of Muslim Youth International INGO NWFP 8,722 Yes Repatriation Up to April 03 10 Medical Corp Mercy Corps INGO Balochistan 181,922 Yes 104,000 8 up to June 03 120 International American INGO Balochistan 157,643 Yes 97,807 8 92 Refugee Committee Association of INGO Balochistan 465,895 Yes 35,835 3 BHUs 72 Medical Referral Doctors of Asia system for all new camps MSF Holland INGO Balochistan - Self- 35,341 2 Approx. funding 45 in Chaman PRCS Nat. Balochistan 30,000 Self 9,000 1 8 NGO funding UNHCR Islamabad 270,324 All Afghan Malaria and external refugees EPI specific procurement Total 5,853,861 1,591,255 118 2165 Source: UNHCR Islamabad

111

Annex 5: List of Acronyms AMDA Asian Medical Doctors’ Association ANC Antenatal Care ARC American Refugee Council ARI Acute Respiratory Infection BHU Basic Health Unit BPRM Bureau of Population, Refugees and Migration CAR Commissionerate for Afghan Refugees CBR Crude Birth Rate CCAR Chief Commissionerate for Afghan Refugees CHS Community Health Supervisor CHW Community Health Worker DANESH Drug and Narcotics and Educational Services for Humanity EMRO Eastern Mediterranean Office (WHO) EPI Expanded Programme of Immunization FATA Federally Administered Tribal Area FHS Female Health Supervisor FCHW Female Community Health Worker FMO Female Medical Officer FSMO Field Supporting Medical Officer (PDH supervisor of health services) GAM Global Acute Malnutrition GoP Government of Pakistan HAC Health Action in Crisis HIS Health Information System HNI HealthNet International IMC International Medical Corps INGO International NGO IRC International Rescue Committee IP Implementing Partner ITN Insecticide treated Mosquito Net LHV Lady Health Visitor MC Mercy Corps MO Medical Officer MoH Ministry of Health NWFP North West Frontier Province OP Operational Partner PDH Project Directorate for Health PHC Primary Health Care PMS Pharmacy Management System PRCS Pakistani Red Crescent Society SAFRON Ministry of States and Frontier Region SFP Supplementary Feeding Programme SGBV Sexual and gender-based violence STI Sexually Transmitted Infection SCF Save the Children Fund SFP Selective Feeding Programme TBA Traditional Birth Attendant TFP Therapeutic Feeding Programme TOR Terms of Reference UNHCR United Nations High Commissioner for Refugees UNICEF United Nations Children’s Fund UNV United Nations Volunteer WAMY World Assembly of Muslim Youth WFP World Food Programme WHO World Health Organization

113

Annex 6: Questionnaire to Follow Up Pilot Interagency Health Evaluations

In 2003 and 2004, several Pilot Interagency Evaluations of Humanitarian Crises Health Programmes were carried out in Pakistan, Nepal and Zambia, as a preliminary activity of the Interagency Health Evaluations in Humanitarian Crises Initiative (IHE-HCI). In July 2004, a Project Coordinator was appointed to manage IHE- HCI activities, including following up on the outcomes of the three Pilot Evaluations. The results and lessons learned from the Pilot Interagency Evaluations will be used in developing a set of guidelines for carrying out interagency health evaluations in the future.

The IHE-HCI Working Group has asked the Project Coordinator to contact agencies involved in the three evaluations, in order to determine which actions were taken as a result, and to obtain feedback regarding the processes and outcomes of the Evaluations. Agencies whose programmes were evaluated in this exercise are kindly requested to complete the attached questionnaire below, providing feedback to the IHE-HCI Working Group.

115 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

PAKISTAN EVALUATION QUESTIONNAIRE REGARDING FOLLOW UP TO RECOMMENDATIONS:

Name: Dr. Naveeda Rehman

Post/Title: Programme Officer (Health)

Organisation: UNHCR

Duty Station: Islamabad

Date: 9 September 2004

To what extent were you involved in the planning of the Evaluation?

1x …………………..5…………………10 low high

"I was not involved in the planning of the workshop. I was informed and asked to coordinate and assist the evaluation team members while they were in Pakistan. The WHO team in Pakistan was not aware of this type of evaluation. We shared relevant material with WHO Representative in Pakistan."

To what extent were you made aware of the results/feedback from the Evaluation?

1……………………5…………………X10 low high

"The results/feedback from the workshop were discussed and shared with me in detail. De- briefing sessions were held at UNHCR Sub Office, Quetta and Islamabad. We discussed major recommendations in detail. Initially a preliminary was shared, and later on the final report was shared."

Evaluation Process:

Which aspects of the preparation/planning of the Evaluation went smoothly?

"No comment, as I was not involved in planning and preparation. There were gaps in the planning: for example, the evaluation team was not briefed on the administrative set-up in place in Pakistan and they were not aware of UNHCR counterparts, line Ministry and department (Chief Commissionerate for Refugees -CCAR and Commissionerate for Refugees (CAR) in respective provinces), working under line ministry. The evaluation team should have had detailed information on health delivery system in the country, stakeholders, role and responsibilities of different actors involved, etc."

Which aspects of planning could be improved upon and how?

"There is a need to involve health, nutrition and HIV/AIDS focal persons in the country in the planning phase. These persons should then be made responsible to coordinate directly with the evaluation team and provide relevant information in advance."

116 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

In what ways did this Evaluation differ from a single agency evaluation?

"This was a more comprehensive, independent and external evaluation. The evaluation team focused on managerial and technical aspects of the programme. IP field staff and refugees were interviewed and their concerns were taken into consideration. Interpreters (Afghan nationals) were hired for a short time to assist the team. The team members tried their best to be transparent, were open, frank and shared information at all levels."

How were the recommendations presented/shared with stakeholders?

Draft recommendations were shared as hard copy with representatives of IP’s, OP’s, UN agencies and Government representatives. De-briefing meetings were held at UNHCR SO, Quetta. IP’s, OP’s, UNHCR, WHO and WFP staff and Government representatives participated. Discussions were held on recommendations. A debriefing meeting was held in Islamabad. Participants profile included: UNHCR BOI and SO-Peshawar staff, IP’s and OP’s from Peshawar, Government of Pakistan representatives from Peshawar and Islamabad. All of the recommendations were discussed in detail. The Interagency Health Evaluation report in hard copy was shared with all IPs, OPs, government services and UN agencies. What, if any, were the constraints to carrying out the Interagency evaluation of the programme?

"The Interagency Health Evaluation went smoothly. WHO and UNHCR assisted the team in terms of sharing information and relevant documents, logistic arrangements, etc. Insecurity was the major constraint in Punjab province, preventing team members from visiting refugees who were located there. Limited time available for the evaluation team was also a constraint to information gathering".

What suggestions can you make to improve the Interagency Evaluation Process?

Ensure participation of the country-level health, nutrition and HIV/AIDS focal staff in planning phases of the evaluation. Improve coordination amongst UNHCR HQ, evaluation team and health staff of the country where evaluations will take place. Develop more detailed TORs jointly with all partners, ensuring participation from stakeholders within the country in addition to Headquarters. Evaluation Outcomes:

Do you feel the conclusions of the Evaluation are relevant to the programme?

"Yes, the conclusions were very relevant; as a result of the Interagency Evaluation, we were able to identify programme strengths, weaknesses, and solutions for improving the situation."

If not, what could have been done differently to improve their relevance?

"More time for the actual evaluation, more preparation beforehand, and a separate budget to cover the costs of implementing the most important recommendations would improve their relevance."

117 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

In terms of recommendations made, were they the right number, too few, too many?

"Too many, especially in light of declining overall resources for the programme in Pakistan."

What steps/actions were taken following the Evaluation to follow up on the recommendations?

Shared the evaluation report with all stakeholders. Presented recommendations to all IPs, OPs and Government representatives. Made hard copies of the report with spiral binding and shared with all stakeholders. Held coordination and follow-up meetings on recommendations with IPs and OPs.

118 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Follow up completed Comments regarding constraints to follow- Overall Recommendations (Check one) up, relevance of recommendations, results YES NO for programme, etc.

The context: Yes We are assuming that substantial numbers - To adopt a health policy that is based on the assumption that the bulk of will remain in Pakistan. We are in the process Afghan refugees will remain in Pakistan. of developing health programmes to bring in other actors e.g. Government of Pakistan line Ministries, UN agencies, e.g., UNICEF, WHO, World Bank, National and International NGOs. The Health policy will be for the people concerned. Might be other issues outside UNHCR’s scope that GoP will take care of (i.e. work permits, etc).

The evaluation: Such a comprehensive evaluation should take - To carry out comprehensive evaluations of such costly health programmes place periodically, keeping in mind the earlier in their lifetime or rather, periodically. changing scenario, environment, and need for - To organize comprehensive staggered evaluations. evolving strategies to cope with diminishing resources. This will help agencies to know where they stand and how to improve their programmes.

Periodic evaluations are preferable, subject to availability of resources.

Assistance strategy: Yes PDH Balochistan was phased out and instead - To improve the current unsatisfactory performance of PDH, either through a small unit called the Health “Liaison Cell” is discontinuing cooperation with PDH as an IP or restructuring/downsizing now working directly under the PDH, or as a minimal option, improving the status quo. Commissioner for Afghan Refugees. This cell - To introduce a formal performance monitoring mechanisms and periodic is not responsible for implementing health evaluations according to criteria elaborated by, and agreed with, by the IPS. activities but acts as a coordination unit.

119 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

PDH-NWFP: Restructuring and downsizing is an ongoing process. - To introduce periodic, formal and transparent tendering processes for UNHCR is in the processes of strengthening contracting out services to IPs in connection with the mentioned monitoring and putting into place a formal performance and evaluation process. monitoring mechanism.

- To move the goalposts in the direction of increased sustainability. New Programme monitoring is improving and objectives must be formulated at a strategic level, in view of prolongation there is closer financial monitoring. (perhaps indefinitely) of the refugee problem). UNHCR is also in the process of developing standard indicators for monitoring and receiving feedback regularly.

In Balochistan Province, tendering processes for contracting out health services were launched. After tendering, a short-listing of IPs was carried out. In such processes, cost analysis was made and financial information requested. Sustainability was not prioritised.

Programme is now focusing on sustainability with the involvement of all stakeholders and bringing in other actors, i.e. line ministries, UN agencies, international and local NGOs. Health Information System:

A rectification process needs to begin immediately, either through a project Yes An HIS rectification process has been put in evaluation or a tripartite review of the collaborative HIS system, by the four place by UNHCR, IRC and IPs. interested parties in Islamabad (CDC, IRC, UNHCR and IPS). All IPs and OPs have a focal person for HIS. The local HIS loop of data collection compilation and local feedback be strengthened. Staffs are trained on new software, which was installed at all IP’s offices.

120 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

IPs and OPs receive feedback regularly. Strengthening of data collection, compilation and data verification is an ongoing process.

Norms and standards: That each BHU have a small reference library of appropriate clinical materials Yes All health units received a clinical reference for use of the health workers, including general reference clinical texts e.g. book—Merck manual, other clinical Primary Mothercare and Population and Primary Childcare references, Emergency Contraceptive That UNHCR assist in NGO in drawing up a pocket book standard guidelines, material on HIV/AIDS, etc.. management manual of common illnesses found in Afghan refugee camps Health Units need to be checked to make sure they are retaining this material at health units.

Pocket booklets on management of common illnesses and on OBS/Gyn are being printed. Some IPs/OP’s have established small reference libraries at health units. Nutrition:

- To continue annual nutrition surveys for monitoring the nutritional status of No Budget constraints affected this activity, which the refugee population. did not take place. - To recruit/second a nutrition technical staff member to assist in developing No Budget constraints prevented the recruitment and training of IPs in growth monitoring and rolling out a monitoring tool. of additional staff. UNHCR is in the process of consolidation and downsizing of the Afghan Refugee programme. - To stop the supplementary feeding programme when the present food supply Yes Supplementary Feeding Programmes stopped has been finished, preferably when the winter is over, but in the meanwhile in all camps in NWFP at end August 2004 look at reintroducing the recommended standard ration with the closure of the new camps, and in Balochistan, by mid-Sept.ember 2004.

- To address the present high levels of chronic malnutrition in Balochistan with No While chronic malnutrition is prevalent a blended food ration as part of the general ration (in particular in the Chaman among old refugee populations, who are not new camps) receiving rations since Sept 1995. there are no food shortages in old camps. By starting

121 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

rations, distribution and targeting the chronic - To supply training tools for identifying and recognizing micronutrient malnutrition cases may create problems. deficiencies and conduct training on micronutrient deficiencies. This activity is planned for the last quarter of 2004. - To consider food support to the chronically ill in the old camp population There is no policy decision on this recommendation. WFP has been approached for nutrition support for TB patients in NWFP.

EPI and child health: - To develop basic standard guidelines for EPI procedures and coverage for all No Monitoring guidelines are being developed. IPs and a simple system for collecting relevant data as part of the HIS Basic standard guidelines on EPI procedures are available but need to be revised, printed and copied for all health units. This process was delayed due to time constraints. This matter was discussed with all IP’s and OP’s. They have developed a list of target To check vaccine wastage and put procedures in place to reduce this wastage. Yes children, a due date list and strengthened out- reach. UNICEF provided standard-dose vaccines for Balochistan province. BCG, DPT and Polio are provided as 20 doses per vial. Vaccine vials Yes with 10 doses per vial are more expensive. To ensure procurement of standard doses of vaccines, considering also other UNHCR procurement for NWFP follows the OPs such as UNICEF for this procurement process. same doses/vial pattern as supplied by UNICEF. Yes Triage is done by health unit staff To introduce triage for all curative care. Yes In NWFP, it is 96.4%, in Punjab—95% and in To make sure that coverage for measles reaches 90 – 95%. Balochistan it is 80%. (security reasons in Balochistan)

122 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Reproductive health: Yes The Reproductive Health Working group has been re-activated at provincial level of NWFP A reproductive health technical support unit should be set up to assist IPs in and Balochistan. These groups meet regularly. setting up and operating reproductive health programmes- Some of the members of the groups provide There is an urgent need for culturally sensitive and tested RH community and technical support to all. health worker education programmes. RH protocols and RH services (many good materials have been developed by some NGS; these could be used more widely.

Afghan males have traditionally made all the decision in RH matters for their Community health workers’ education families. RH initiatives for Afghan refugees need to emphasise male education programme is an on-going activity. Education and involvement. sessions targeting males are held on various RH topics in refugee camps.

Curative services: IPs need to have a clinical supervisor who can assist in setting up more Yes Medical Coordinators, Supervisors and Field efficient patient flow mechanisms in the BHUs, provide clinical supervision of Supervisory Medical Officers are carrying out doctors working in the BHUs and provide some in-service training from time clinical supervision and on-the-job training. to time-

Patient flow process through the BHU should begin with triage or screening of No Done clients for those who are very ill and need priority attention Training on the rational use of drugs is a priority. No UNHCR plans to carry out this training in collaboration with WHO. UNHCR SOs will also discuss this matter with the Core Committee on training. Every camp needs a community centre where people can meet, sport be Only very few camps have community encouraged and entertainments held. centres, where staff meet with refugees. These centres provide skills training to women and adolescent girls and boys. Medical supplies: To take the drugs procurement into UNHCR’s own hands and to centralize it Yes UNHCR achieved this target by centralized in Islamabad. procurement.

123 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

Training: -To set up a core group to identify present training being conducted, training Yes A core committee on training has been set-up tools available and to streamline training according to the needs and is responsible for coordinating training. Master Trainers are available with IP’s with -To improve the recruitment process of trainers, employing personnel with Yes appropriate language skills and take care of relevant technical competencies; trainers also need to have appropriate training refugees. UNHCR SO Peshawar is in language skills when training refugees. the process of evaluating the technical knowledge and language skills as a part of rationalisation process. Gender-related issues: Yes No As far as possible, the IPs and OP’s recruit -To actively recruit female staff to vacant health posts and address their female staff. UNHCR SO, Peshawar is re- operational difficulties for working in rural areas. enforcing this arrangement with PDH-NWFP.

All IP’s and OP’s are trying their best to -To improve the ratio of female to male community health workers Yes improve ratios of female to male health workers. With some of the IP’s the female health workers are outnumbered compared to male health workers.

The most difficult target to achieve is PDH- -To develop more innovative social programmes particularly for female Yes NWFP. Community services are engaged on refugee as an outlet, at present the BHUs appear to be the main/only meeting such programmes for female refugees. Some or socialising place. of the IP’s have started loan-lending schemes for refugees for starting small businesses. Protection: -To gather systematically and in a multisectoral effort data on SGBV in order to Community services took the lead for know the magnitude of the problems and to explore potential ways of tackling coordination of all activities on SGBV. them in cooperation with local partners A data collection form was developed and shared with all IPs on health, education, community services, protection and water/sanitation.

UNHCR and IP’s staff were sensitised and

124 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

trained on SGBV. Data collected by all IPs are shared with -To improve the balance between female and health workers at all levels Yes UNHCR protection sector for information, while maintaining confidentiality, Protection -To develop short training courses tailored for staff at different levels, of both Yes sector provided assistance. OPs and IPs, in order to raise awareness about protection issues and On-going process procedures/protocols Community services designed training course, developed guidelines and imparted training in the field. Health, Community Services and Protection Sectors planning to train IP’s staff in October in NWFP. Community services in Balochistan is engaged in training of staff on SGBV. UNHCR representative in Pakistan has nominated two female gender focal points at each UNHCR Office in Pakistan, but there is still a lot to do in this field. Referral: -To commission locally an evaluation of the functioning of Chaman Hospital Yes An evaluation was carried out by UNHCR and to share findings with the main stakeholders (including the locally active and AMDA (International NGO) and set up NGS) in order to establish a strategy for revitalising it for as long as the camps the mechanism of assisting this hospital. The exist assistant will be phased out by mid- Sept. 2004. Appropriateness and Participation: -To revive the attempt at increasing materials participation through user fees Yes UNHCR is collaborating with IP’s in the (linked of course to an increased participation in decision-making) process of reviewing community participation guidelines. Once this is completed, UNHCR will seek input from communities. The system will be in place next year.

The UNHCR Resident Auditor is of the view

125 JOINT EVALUATION OF HEALTH AND HEALTH PROGRAMMES FOR AFGHAN REFUGEES LIVING IN PAKISTAN – DECEMBER 2003

that either the community should keep these funds or if UNHCR has these funds, there is a need to develop a proper system to report, closely monitor, and maintain financial checks and balances. Effectiveness: -To measure effectiveness not only in input, process and outputs indicators Yes Survey and evaluations are the tools to find (e.g. no. of staff, training provided and patents seen, respectively) and to make out treatment success, behaviour changes and an effort to measure more systematically health outcomes, such as treatment impact of planned activities. success, behavioural change and knowledge gains. Impact indicators provide such information on a limited scale, whereas further Interagency Evaluations will fulfill the purpose at large and provide useful information for programme planning and strategy revision.

Connectedness and Integration: -To discuss the current and future impact of budget cuts in a transparent way Yes UNHCR is in the process of coordinating with with IPs and donors and to look at alternative assistance strategies that take a MoH, NGOs and UN agencies. wider time horizon into account. -To include the MoH policy considerations and a partner in the planning IP’s provide information on other sources of process. funding. Coverage: -To include the urban refugees into UNHCR’s health policy framework. No UNHCR’s financial constraints do not permit it to expand services to urban refugees. Efficiency: -To strengthen UNHCR’s technical/managerial capacity by filling the position Yes The placement of Health Coordinators (UNV) in Peshawar with a senior public health person, preferably an expatriate with a in Peshawar and Quetta has strengthened 3-year mandate A focal point for health is also needed in Quetta. UNHCR’s technical/managerial capacity. To improve efficiency in various areas as suggested in Chapter 6, 8 and to include efficiency indicators in routine monitoring and evaluation.

126