MSF Holland Afghanistan Evaluation Internal use only

MSF Holland in Afghanistan Mission Evaluation: May 2000-May 2002

Fiona O’Reilly & Jeremy Shoham

1 MSF Holland Afghanistan Evaluation Internal use only

Table of Contents

Summary 4

1. The Evaluation 7

1.1 Purpose 7 1.2 Methodology 7 1.3 The structure of this report 7

2. Overview 8 2.1 Food security, nutrition and health 8 2.2 Severe drought 8 2.3 WFP VAM assessment 9 2.4 UN appeal for 7.5 million “hungry” Afghans 9 2.5 Low levels of acute malnutrition 10

3. MSF Holland: changes in approach over time 13 3.1 From the rural areas to the cities 13 3.2 Drought response 13 3.3 Reducing Humanitarian space 13 3.4 Monitoring effects of the Drought 14 3.5 September 11th 14 3.6 Re-entry post September 11th 14 3.7 Monitoring Repatriation 15

4. Programmes Ongoing at start of Period evaluated (May 2002) 16

4.1 Western Districts Primary Heal Care programme (PHC) 16 4.1.1 Conclusion 16 4.2 Emergency Preparedness and Response (EPR) Kandahar 19 4.2.1Early Warning System 19 4.2.2 Infectious Disease Ward (IDW) 20 4.2.3 Conclusion 21 4.3 Bala Morgab, Badghis, PHC 22 4.3.1 Conclusions 22

5. Programmes started in 2000-2001 24

5.1 Emergency Drought Response 24 5.1.1 Incorporated nutritional surveillance activities 24 Food Crisis Monitoring 25 MUAC monitoring 25 Conclusions 27 5.1.2 SFCs in selected communities 28 Conclusions 31 5.1.3 Blanket Feeding Distribution 31 Conclusion 33 5.1.4 Mass measles campaign 33 5.1.5 EDR Conclusion 34

5.2 MSF in Internally Displaced Persons (IDPs) Camps 35 5.2.1 Shaidai 35 5.2.2 Maslakh camp 35 Surveys 37 Advocacy 38 Conclusion Shaidai and Maslakh 39

2 MSF Holland Afghanistan Evaluation Internal use only

5.3 MSF's involvement in the Paediatric Ward, Herat 40 5.3.1 Regional Therapeutic Feeding Centre (RTFC) 41 5.3.2 Conclusions 43

6. Period September to November 2001 44

6.1 Evacuation 44 6.2 Regional Approach 44 6.3 Conclusions 45

7. Programmes Started 2002 46

7.1 Emergency support to PHC in Badghis Province 46 7.1.1 Conclusion 46 7.2 Repatriation of Returnees 47 7.2.1 Conclusions 47 7.3 Chaman and Spin Boldak camps 48 7.3.1 Advocacy and co-ordination 48 7.3.2 Conclusions 49

8. Advocacy 48

8.1 Conclusions 51

9. MUAC versus Weight for height 53

9.1 Conclusions 55

10. Cross cutting Conclusions and Recommendations 56

10.1 Cross cutting Conclusions 56 10.1.1 Appropriateness of response 56 10.1.2 Assessment and analysis 56 10.1.3 Programmes serving multiple purposes 57 10.1.4 Approach to implementation in an ongoing crisis 57 10.1.5 Decision Making Process 57 10.1.6 Health programmes 58 10.1.7 Nutrition programmes 58 10.1.8 Programme closure 58 10.1.9 Advocacy 59

10.2 Recommendations 60 10.2.1 MSF General 60 10.2.3 MSF Nutrition Division 61 10.2.4 MSF Health Division 61 10.2.3 MSF Context /Advocacy 62 10.2.4 MSF Afghanistan Programme 62

3 MSF Holland Afghanistan Evaluation Internal use only

Summary

Purpose

This report is an evaluation of MSF activities in Afghanistan between May 2000 and May 2002. These activities have mainly comprised nutritional and health programmes. The evaluation set out to i) understand MSF’s role in Afghanistan and how MSF adapted nutritional and health programmes in a rapidly changing political and operational context in the cultural setting of Afghanistan ii) examine how MSF used its proximity in the region to witness violations of humanitarian principles and law and how MSF positioned itself in its advocacy work.

Methods

Two external consultants with backgrounds in nutrition and health undertook the evaluation. MSF Headquarters staff briefed the consultants at the start of the evaluation. Field work took place in to Kabul, Herat and Kandahar where interviews were carried out with MSF project staff , personnel from INGOs, local NGOs, UN agencies, and MoPH staff. The consultants also attended a regional workshop in Ashgabad and a medical co-ordination day in Amsterdam. Phone interviews were conducted with staff in Quetta, Pakistan while a number of MSF staff no longer involved in the programme were interviewed in London and Amsterdam.

A large volume of documentation and email correspondence was reviewed. Limitations included lack of a central repository for reports and information on the programme which meant that the consultants had to sift through scores of documents to find key information. At field level much information was lost or misplaced during the evacuation. The high staff turn over meant that most MSF international staff connected to the programme over the time period in question were no longer in Afghanistan at the time of the field visit nor were many key personnel from other INGOs and UN agencies present during the period under question.

Attempts have been made to counter the effects of these limitations through accessing key personnel through other means (e.g. co-days interviews, telephone interviews, email questionnaire). However the result has been that some MSF projects are reviewed more comprehensively than others reflecting the amount and quality of information available. Where indicators for effectiveness are available they have been assessed against accepted technical standards. Where such data were not available projects have been judged on the basis of ‘impressions’ as to whether objectives and intended outcomes were achieved.

Findings

In view of the extremely difficult working environment in Afghanistan over the period evaluated MSF’s achievements have been impressive. MSF was involved in three provinces of Afghanistan, Badghis Herat and Kandahar. Programmes included, hospital based medical activities, emergency health and nutrition services in displaced and refugee settings and transit sites and an early warning system in sentinel sites in Kandahar. Nutritional activities included food security assessment nutritional surveillance as well as blanket and selective feeding programmes. During the period in question MSF responded to disease outbreaks swiftly and effectively, implemented thoughtful and comprehensive primary health care programmes and vaccinated thousands of children against measles.

Within a context of the appalling health status of women and children and poor access to basic health facilities in Afghanistan, the focus of MSF’s operational activities was affected by what the Taliban would allow, security , a developing drought induced food crisis, lack of reliable information and how MSF perceived its role (i.e. whether it should focus on needs arising out of crisis, human rights monitoring, or health needs of the most vulnerable). In the event over the time period in question focus changed from improving health care access in rural areas to focusing on health needs in cities and eventually to safe motherhood in remote areas. The constraints under which MSF operated cannot be over stated.

MSF’s primary analysis of the impact of the drought was in keeping with many other agencies, i.e. that there was a developing food crisis, which required emergency food and nutrition interventions.

4 MSF Holland Afghanistan Evaluation Internal use only

Although MSF’s perception was that a General Food Distribution (GFD) was the most appropriate response, staff were conflicted about whether MSF should take on an implementing role. Ultimately the decision was taken (based on many factors and previous experience) that MSF would not ‘do’ GFDs. MSF therefor focused on advocating for a GFD and later implemented Blanket Food Distributions (BFDs) as stop gap measures. MSF implemented traditional emergency feeding programmes in a number of open settings partially to collect data which would be useful in advocacy but also to prevent severe malnutrition. However, there was limited evidence of alarming malnutrition prevalence and traditional feeding programmes were found to be inappropriate to the cultural setting eventually these were either discontinued or adapted (e.g. mobile SFP).

Thorough investigation of outcomes in terms of malnutrition and mortality was not rigorously undertaken by MSF largely due to the constraints and limitations imposed by the authorities as well as cultural factors and the harsh climate. However where other agencies managed to undertake thorough representative investigation mortality was found to be high and related to disease while levels of malnutrition were lower than expected in the context of a food crisis.

There was a considerable need within Afghanistan for credible information on how the drought was impacting the health and nutrition of the population. As a health agency with historical capacity in nutrition MSF have capitalised more on their comparative advantage in the health sector by analysing and highlighting the health component of the crisis and by focussing their interventions more fully on the health sector, i.e. investing more resources in further strengthening and expanding health programmes (EPR and PHC) and integrating nutrition components (where necessary) into these health programmes. Also MSF was well placed in the regional paediatric ward to strengthen and support health information systems and analysis and strengthen links in referring clinics, however this was not done. Given MSFs current core competencies, this may have been a better option than focussing on a sector (food aid) where MSF were dependent upon the commitment and response capacity of other agencies. This does not negate the validity of advocacy actions to improve food aid.

MSF placed too much confidence in rapid MUAC assessments which it saw as a viable alternative to weight for height surveys. Similar prevalences of Global Acute Malnutrition (GAM)were not identified in a limited number of surveys reviewed where both indicators were measured. The correlation between weight for height measurements and MUAC requires further analysis in this setting before MUAC can be used with confidence in place of weight/height to define prevalence of malnutrition. Trend analysis was also affected by assessments carried out on changing populations and using varying age and height cut-offs.

MSFs programmes were often expected to serve multiple purposes (to monitor human rights, to provide information for advocacy, to provide a strategic location for teimonage, to respond to the health needs of the most vulnerable). These purposes were sometimes conflicting.

The changes in approach, although related to the changes in context were also heavily influenced by changes in personnel, i.e. HOMs and ODs with differing interpretations of the needs and MSF's role. This led to contrasting strategies during the period covered by the evaluation e.g., a focus on cities versus rural areas, a focus on advocacy versus technical efficacy, and a focus on working through local partners or independently.

MSF did not have policies which addressed some of the issues that arise in connection with working in the type of longer-term health crisis that exists in Afghanistan. Such a context raises issues on project management, how to work with, through or independently of local partners and infrastructure, the degree of dependence on expatriate staff, coherence of health strategies and methods of withdrawing from programmes. Arguably, this absence of policy led to a modus operandi that was not always appropriate for the context.

MSF embarked on a comprehensive advocacy role in Afghanistan. Although much of the advocacy work was important, there were also examples of failed advocacy or where effectiveness was constrained by the process and method used to advocate. For example, MSF were largely not effective in mobilising GFDs in areas where they were implementing BFDs.

5 MSF Holland Afghanistan Evaluation Internal use only

Key recommendations from this evaluation include the following

• MSF should clarify its role vis-à-vis the primacy of health, medical and humanitarian aims in a way which supports field managers to identify appropriate MSF interventions. This should be reflected in the MTP while its impact should be monitored in subsequent programming.

• The MTP should also clarify MSFs role in chronic health emergencies as well as its position with regard to working with and within national health infrastructure.

• In countries where a longer-term presence is likely and desirable MSF should investigate models of National staff development to reduce MSF HRM constraints e.g. investigate potential for project management positions to be filled by National staff.

• MSF should elaborate its policy on when it will engage in activities which are not core competencies i.e. in particular food, shelter, water and sanitation, to give a clear picture of what constitutes ‘dire need’. In this regard MSF should clarify its policy and strategy on implementation of general food distributions based on an analysis of past experience and the internal debate on this issue that has taken place over the past decade. This clarified policy should set out options on what to do in circumstances where food insecurity is severe and undermining MSF health programmes

• The process of approving technical proposals should include approval from the relevant technical department. Objectives should be explicitly stated in proposals even where these do not relate directly to health activities.

• The link between quality assurance and experienced and skilled staff should be recognised. MSF should explore other means to ensure availability of suitably experienced nutrition staff. If MSF want to maintain nutrition credibility they may have to review methods of attracting and retaining skilled and experienced specialist personnel e.g. institutionalising some form of apprenticeship or mentoring scheme, review of allowances in comparison to other volunteer agencies.

• MSF should increase capacity to support epidemiological analysis in areas of operation as well as competence in integrating food security, health status and nutritional status analysis

• The process of advocacy should seek to understand constraints and influence before resort to defamation and confrontation. Advocacy agendas should arise out of operational activities and be formulated by field personnel with support from HAOs and HQ staff. Advocacy issues should always be supported by credible data.

• Technical coordination within country should be encouraged by HQ and in training.

• Unless the current consensus changes (as evidenced by new research) MSF should treat MUAC assessments and surveys with the appropriate caution.

• MSF Afghanistan should develop a longer term country health policy and strategy (3 years) in collaboration with Amsterdam. This should clearly outline MSF's main focus in the country, d where and how MSF will work, and how MSF will relate to indigenous health infrastructure. Both MSF's humanitarian and health role should be explicitly addressed. The policy should include an emergency preparedness and response component which would identify what type of emergency response capacity MSF will maintain covering areas such as food crisis response.

6 MSF Holland Afghanistan Evaluation Internal use only

1. The Evaluation

1.1 Purpose

The purpose of the evaluation was to learn about MSF’s role and how MSF adapted nutritional and health programmes in a rapidly changing political and operational context in the cultural setting of Afghanistan. At the same time the evaluation was to examine how MSF used its proximity in the region to witness violations of humanitarian principles and law and how MSF positioned itself in its advocacy.

1.2 Methodology

The focus of the evaluation was on nutritional and health programmes between May 2000 and May 2002.

This evaluation was conducted by two external consultants with backgrounds in nutrition and health. The evaluation was conducted through: • interviews with MSF project and HQ staff , personnel from INGOs, local NGOs and UN agencies working in Afghanistan and MoPH staff in Kabul, Herat, and Kandahar • review of documentation • regional workshop and attendance at the 2002 medical co-ordination day • field visits to Kabul, Herat and Kandahar • phone interviews with staff in Quetta, Pakistan

Both consultants were briefed in Amsterdam at the beginning of this evaluation over a period of three days. The principal evaluator subsequently spent 4 days in Ashgabad at a regional workshop and 14 days in Afghanistan visiting project sites and interviewing country programmes staff from MSF and a number of other agencies.

Limitations: • no central repository for reports and information on the programme within MSF meant that the consultants had to sift through scores of documents to find key information. It is by no means certain that all key documents/surveys were made available to the team • much historical documentation at field level was lost or misplaced during the evacuation • high staff turn over meant that most MSF international staff connected to the programme over the time period in question were no longer in Afghanistan at the time of the field visit • the two week field visit did not include MSF operations in Badghis • personnel from other INGOs and UN agencies present during the period under question were often no longer present during the field visit • programmes were no longer ongoing (e.g. EDR in Kandahar, Western districts)

Attempts have been made to counter the effects of these limitations through accessing key personnel through other means (e.g. co-days interviews, telephone interviews, email questionnaire) However the result has been that some MSF projects are reviewed more comprehensively than others reflecting the amount and quality of information available. Where indicators for effectiveness are available they are assessed against accepted technical standards. Where effectiveness data were not available effectiveness is judged on the basis of ‘impressions’ as to whether project objectives and intended outcomes were achieved. As this is a retrospective review of what was done (in contrast to a real-time evaluation where constraints and context would be better understood) the evaluation team has ‘pulled-back’ from being overly critical or proscriptive. With a view to primarily strengthening institutional memory the report has therefore placed most emphasis on trying to capture what happened, why it happened and what lessons can be learnt.

1.3 The structure of this report

The report is organised by MSF project with specific conclusions drawn for each discrete project activity. The evolution and justification for each project/programme is described and where possible effectiveness, appropriateness and connectedness is assessed on the basis of available information.

7 MSF Holland Afghanistan Evaluation Internal use only

2. Overview

The Taliban (meaning religious student) formed and took control of Kabul in 1996 expanding to control around 90% of the country. Various factions constituted themselves to oppose the Taliban and occupied areas in the north of the country. Fighting continued between the Taliban and these factions causing displacement and increasing the numbers of refugees to Pakistan throughout the middle to late 1990s1.

On September 11th,2001, the USA sustained unprecedented terrorist attacks on the World Trade Centres and Pentagon buildings and on October 7th initiated military strikes against the Al Quaeda terrorist network in Afghanistan and against the Taliban. Throughout October, November and early December, these military strikes continued. The Northern Alliance and non-Taliban Pashtun groups rapidly advanced and captured Afghanistan’s major cities and towns. By early December, the last stronghold of the Taliban, Kandahar, fell and the Taliban was defeated. An interim Government was installed to lead the country in December 2001.

2.1 Food security, nutrition and health

Afghanistan has a population of around 22 million and is made up of five major ethnic groups2. An estimated 12 percent of land in Afghanistan is suitable for arable farming. In 1975 the country was self sufficient in wheat with irrigated land accounting for over 80 per cent of agricultural production. Since the onset of war, however, this situation has been reversed with a growing dependency on food imports and food aid. Sources of income include labour, remittances, trade, handicraft (carpet production) and sale of livestock. Many also depend on the cultivation of and trade in opium.

The war has impacted on both the production and purchase of food, either by direct effects such as the destruction and mining of fields and roads, or indirectly through siege tactics (or blockades) which were a common strategy used by the Taliban. Those most affected include the population groups perceived to be supporting the Northern Alliance (mostly in the north, north-east and Hazarajat)3.

2.2 Severe drought

Following severe drought in parts of the country in 1999 and countrywide in 2000, Afghanistan suffered a third year of widespread and severe drought in 2001. A joint FAO/WFP crop assessment reported that rain-fed and irrigated crops had almost totally failed and that all population groups had limited access to food. Livestock decimation was reported with “catastrophic livelihood consequences” for the Kuchi (nomads) and with serious effects on livestock-holding farmers. Evidence of pre-famine indicators and increases in the number of IDPs and refugees led the UN to recommend an urgent international response to “avert imminent catastrophe”4. A survey5 in 13 provinces found widespread evidence of the impact of the drought in terms of indebtedness, depletion of assets, reduction in food consumption, and shortage of water.

1 Johnson, 1998 2 The five major ethnic groups ate: the Pashtuns, the Tajiks, the Uzbeks and the Turkmen, and the Hazaras. There are also the nomadic and semi-nomadic pastoralists, ‘Kuchi’. The Tajiks, Uzbeks, and Hazara were perceived to be opposition supporters by the Taliban (mostly Pashtun) and were subjected to blockades, destruction and forced displacement. 3 Jaspars, and Fielding, 2002, January 4FAO/WFP, 2001, June 5 A Cash Famine. Sue Lautze.

8 MSF Holland Afghanistan Evaluation Internal use only

TABLE I. LOSS OF SECURITY AMONG SURVEY RESPONDENTS IN 13 AFGHAN PROVINCES, MARCH 1999 – 6 MARCH 2002

Category % of % of % of Percent Change Respondents Respondents Respondents (1999/2000 to Secure Secure Secure 2001/ 2002)‡ 1999/2000 2000/2001 2001/2002 Debt 56% 21% 14% 75% Decrease Diet 59% 17% 9% 85% Decrease Assets 41% 13% 13% 70% Decrease Water Resources* 43% 14% 15% 65% Decrease ‡ Errors due to rounding *Water available for agriculture. Excludes Kabul.

Households had lost control over water resources for a range of reasons including the fact that some wealthy households were draining water from the poor by drilling deep wells and using powerful pumps. This hastened the depletion of the water table in the area, causing the surrounding shallow wells to fail.

2.3 WFP VAM assessment

Following the FAO-WFP crop assessment WFP and partner NGOs carried out the first country-wide food security assessment between July and August 20017. The assessment estimated crop production and other sources of food and income. The proportion of the population to be targeted was based on previous food economy baseline information (the proportion of landless labourers and farmers with small landholdings) and informed guesswork.

The estimate of numbers in need of food aid rose from 3.8 million8 to 5.5 million people (25 percent of the total population) following the assessment with more than one million people displaced in the 12 months preceding the assessment.

The assessment was criticised for underestimating both the nature and the distribution of needs in part because of limited capacity for data analysis but also because of the difficult operating environment as well as an arguable level of donor apathy and antipathy relating to political relations with the Taliban. Some highly vulnerable areas were not identified by the assessment and, as a result, never received adequate prioritisation of assistance.

2.4 UN appeal for 7.5 million “hungry” Afghans

Following the September 11th attacks, the UN estimated that 2 million Afghans above the 5.5 million were in need of food aid. This was based on assumptions about the likely numbers who were vulnerable to internal displacement or to becoming refugees in neighbouring countries. The new emergency programme aimed to reach a total of 7.5 million Afghans: 6 million in Afghanistan and 1.5 million refugees in the surrounding countries (Pakistan 1 million, Iran 400,000, Turkmenistan 50,000 and Tajikistan 50,000). WFP estimated that 355,000 MT of food aid (mainly wheat) would be needed

6 While Table 1shows a steep decline over the past three years, the largest jump occurred between 1999/2000 and 2000/2001. For example, the numbers of secure households fell by between 60% and 70% with respect to diet, debt, asset and water security between the first and second year of the drought. It is also important to remember that the latest change in the political regime occurred halfway through the third Afghan year (March to March) in the table, and the results therefore do not clearly illustrate the more recent positive (or negative) changes in security. 7 The VAM assessment did not include IDPs, nomadic or urban groups. 8 By early 2001, WFP were providing food aid through bakeries, food for work, school feeding, food grants for returning refugees and food for asset creation programmes for 3.8 million Afghans.

9 MSF Holland Afghanistan Evaluation Internal use only in Afghanistan over a six-month period (October 1st to March 31st) and that it would aim to deliver approximately 52,000 MT per month9.

Immediately after the attacks, WFP suspended its cross border deliveries of food aid into Afghanistan and resumed the programme 19 days later. Nonetheless, local WFP staff and partners reported that they had continued to distribute existing grain stocks to an estimated one million people through September.

Data from a variety of sources shows a massive increase (from just below 9% to over 60%) in the percentage of households receiving food aid during the period from March 1999/2000 to March 2001/2002. The sharpest increases have been over the past year, with the percentage of households receiving food aid increasing by approximately 300% (from just under 20% to just under 60%).

10 FIGURE 1. PERCENTAGE OF HOUSEHOLDS RECEIVING FOOD AID, 1999 – 2002

Households Receiving Food Aid

100.0%

90.0%

80.0%

70.0%

60.0%

50.0%

40.0%

30.0%

20.0%

10.0% % of Households in Survey % 0.0%

March 1999- March 2000 March 2000- March 2001 March 2001- March 2002

2.5 Low levels of acute malnutrition

Nutrition information representative for the whole of Afghanistan is limited although some local nutrition surveys were carried out between 2000. These surveys did not report unusually high levels of acute malnutrition (see figure 2)11. The relatively low levels of malnutrition contradict the food security information from the FAO/WFP crop assessment and the WFP/NGO food needs assessments. Some nutritional surveys also show very high crude and under-five mortality rates, which cannot be explained by either the malnutrition or food security levels.

9 The standard WFP ration for Afghanistan, until January 2002, was 50 kg wheat/family/month. A family is estimated to have 6 members. In January 2002, 5 litres of oil and 12.5 kg pulses, and in some cases 25 kg blended food was also added in the most food insecure areas in the northern region. 10 S. Lautze, A Cash Famine February 2002 11 A rapid nutrition survey carried out in March 2001 in the north found overt cases of scurvy (vitamin C deficiency) and cases of angular stomatitis suggesting a complex of vitamin deficiencies.

10 MSF Holland Afghanistan Evaluation Internal use only

12 FIGURE 2. PREVALENCE OF ACUTE AND CHRONIC MALNUTRITION SURVEY RESULTS, 2000 – 2002

2000-2002 Nut survey results

80 70 60 50 Acute Mal 40 Chronic 30 20 10 0

NE Feb-00 Mazar Nov-00 Khosh Sep-01 Mazar Mar-02 Faryab Aug-01 Badghis Feb-02 Maslack Apr-02 Herrat city Jul-00 Kabul city Oct-00 Kohistan Apr-01 Belchirag Apr-02 Kabul cityKandahar Feb-00 May-00 Jalal AbadKabul Dec-00 city Mar-01 Fayz Abdad Sep-00 panjsher/som Aug-00 panjsher/som Mar-01 Kandahar city Aug-01

There are five possible explanations for the discrepancy between food security information, malnutrition and mortality rates: • Food insecurity was over-estimated: Much of the income available to households in Afghanistan may be difficult to measure precisely. In particular, remittances are known to be an important source of income, as is informal trade, and the production of illegal crops such as opium. Furthermore, borrowing and taking out loans are common coping strategies in Afghanistan. • Many of the nutrition surveys have focussed on urban centres where food insecurity may not be as severe and where some health infrastructure is present13. • A number of surveys note that children are preferentially treated and adults withheld food to protect their children. This was particularly obvious in a Concern survey in Badakshan that reported 21 percent of mothers were malnourished (MUACs < 21.5 cm)14 • Communicable diseases including measles, diarrhoea and respiratory diseases were reported to be the main cause of death in surveys carried out in 2000 and 2001. Alarming levels of mortality for the North of the country, at 2.6/10,000/day (Faryab Province) and 2.3/10,000/day (Panjshir Valley) suggesting an “emergency situation out of control” were recorded. In some rural areas health services are non-existent and access to water and sanitation are extremely limited15

12 UNICEF, Kabul 13 In the opposition held areas of the Panshir Valley, Faryab and in the Khosh Valley where drought and conflict are more severe and where large numbers of people have been displaced, the levels of acute child malnutrition were found to be higher than those found in the urban areas. 14 UN SCN, 2001, October. 15 UN SCN, 2001, July

11 MSF Holland Afghanistan Evaluation Internal use only

• Micro-nutrient deficiencies may have caused the high rates of mortality. Micronutrient deficiencies have not been widely documented but a scurvy outbreak during the winter of 2000-2001 in Faryab indicates that micronutrient status is marginal in some parts of the country and that dietary quality should not be overlooked 16

In spite of the massive relief effort significant proportions of the population remained food insecure at the end of the period under evaluation. The Afghan population over recent years has been overwhelmed by successive hazards, which has reduced ability to cope with loss of agriculture and livestock unemployment and burgeoning debt burdens10.

16UN SCN, 2001, April

12 MSF Holland Afghanistan Evaluation Internal use only

3. MSF Holland: changes in approach over time

3.1 From the rural areas to the cities

At the start of the period evaluated MSF were changing programme emphasis. MSF had been engaged in Primary Health Care in three districts in western Herat in partnership with a local NGO (CHA) since 1997. The programmes provided comprehensive health care with an emphasis on maternal and child health. The programme was also near the Iran border thereby providing an opportunity for ‘temoinage’ and was rurally based thereby minimising interference from the Taliban. However the relationship between MSF and CHA had effectively become one of donor and recipient. MSF were basically providing management, logistical and resource support. Following a field trip in 1999 the new Operational Director (OD) felt that programmes had lost overall focus and did not fit well with MSF’s role and mandate. The health needs the programmes were addressing were not necessarily needs which arose out of the crisis situation, but were ‘normal’ /usual health needs. The increasingly restrictive behaviour of the Taliban towards foreign NGOs also played a role in re-orienting MSFs strategy at this time. The Taliban were extremely suspicious of MSF's work and restricted staff movement and access to beneficiaries. It was an extremely compromising situation to work in. Furthermore, the new OD believed that cities may have been more affected by the evolving crisis than the rural areas which had never had good access to health. A role for responding to health needs arising out of crisis rather than a ‘normal’ situation is elaborated on in the MTP.

These factors contributed to MSF’s decision to begin working in the paediatric ward in Herat City and in the IDW in Kandahar. The continued presence in Herat allowed ongoing monitoring of the returnee process from Iran. The involvement in the IDW was also a logical follow on from a cholera response MSF made in 1999. It was at this time that MSF also started working in an isolated area of Badghis providing PHC in Balamorgab district. This arose as MSF wished to follow a community who they had been providing services for in Shaidai IDP camp in previous years. This population which had been displaced due to conflict were returning home to areas where health services were non existent.

Part of the strategy for working in the IDW and PW was to use the wards as locations from which further needs could be identified in the districts and activities accordingly extended. Also the opportunities to strengthen referral systems in the regions would have a significant health impact. An EWS was established in 17 sentinel sites in Kandahar districts in conjunction with the IDW. These sites were clinics run by Local NGOs (AHDS and IbnSina). Overall, these higher profile operations created more opportunities for dialogue with the authorities.

3.2 Drought response

As MSF was engaging with these new activities the effects of the drought were becoming evident. Food security agencies and WFP who had carried out rapid assessments were raising the alarm and predicting a food crisis. ACF who had been systematically surveying the major cities reported low levels of acute malnutrition but a serious chronic nutrition and mortality situation. In 2000 ACF were still of the view that the existence of malnutrition was more related to the health water and sanitation situation than access to food. MSF conducted assessments in its areas of operation (Herat, Badghis and Kandahar) and confirmed WFPs analysis of an imminent food crisis. As a result of this assessment MSF launched an Emergency Drought Response. The focus of this response was to establish a nutrition and food security surveillance system using its position in the health clinics in the districts but also though the establishment of SFCs. Around the same time the influx of IDPs into Herat began. People were leaving Badghis, Ghor and Faryab due mainly to drought and the exhaustion of coping mechanisms. ICRC initially commenced responding to IDPs in Herat while gradually UNOCHA and others became involved. MSF responded by providing basic health services in Shaidai camp and an SFC. When Maslack opened MSF provided emergency health services in the transit centre and eventually opened SFCs and a TFC. Data from the IDP camps showed that nutritional status of the new arrivals had not significantly deteriorated and that people were probably leaving before winter set in while they had an opportunity to leave and obtain assistance.

3.3 Reducing Humanitarian space

The monopolisation by the UN of the co-ordination mechanism began to raise MSF concerns. The UN had a clear position with regard to the Taliban (sanctions were being enforced) and MSF felt that a

13 MSF Holland Afghanistan Evaluation Internal use only humanitarian effort ‘co-ordinated’ but more likely steered by the UN would compromise humanitarian principles of impartiality and neutrality. The UN initiated a framework for humanitarian response in Afghanistan (SFPCP). MSF took a principled stance with regard to this framework and decided not to be part of it and to speak out for humanitarian principles and against mechanisms which would undermine them.

3.4 Monitoring effects of the Drought

While the Emergency drought response was established quickly in Kandahar, it happened more slowly in Badghis and Herat. Wide scale measles vaccination campaigns were undertaken in the districts of Kandahar with more limited campaigns in Heart and Badghis. Three SFPs were established in Kandahar Herat and Badghis. Monitoring information was crucial to the implicit objective of the EDR- to advocate for a GFD. To this end SFC an sentinel sites were to collect analyse and monitor nutrition and food security data. MUAC measurements were taken in the districts in conjunction with the measles vaccinations and later through the BFD sites. Food security and population movement data were also collected where possible.

By mid 2001 the effects of the drought had reached crisis point. Field teams were demanding that the situation had truly reached ‘emergency proportions’. A modification was made to the EDR, which included the provision of BFD in selected areas. This programme was implemented partly as an advocacy strategy to encourage full provision of general rations to most affected population and as a holding operation until general rations were distributed. This strategy was implemented between May and August in Kandahar.

3.5 September 11th

By September 2001 the situation was looking increasingly dire. The WFP appeal for food aid had not been fully met, and evidence pointed to complete erosion of coping mechanisms. There were reports of outbreaks of micronutrient deficiency disease like scurvy in some isolated rural areas. Then September 11th occurred and the landscape dramatically changed. The entire international presence disappeared overnight from Taliban held Afghanistan. Fears of a massive outflow of refugees into neighbouring countries were very high. MSF reinforced and expanded its presence in Pakistan. Missions in neighbouring countries and other sections collaborated to provide a regional response. Stock piling of resources, personnel and equipment in neighbouring countries began. MSF was positioned and ready to go back in as soon as possible. Inside Afghanistan, the Herat and Badghis programmes continued and the approach was to support these from Ashgabad. In Kandahar after some time operations came to an end and the office, house and stores were looted. MSF began working in the staging areas (Killi Faizo) in Pakistan for refugees (mostly Pashtun) and in IDP camps near the Pakistan border (Spin Boldak) providing health, nutrition and water and sanitation services and advocating for full provision of services to refugees. During the Sept to Nov 2001 period there was a strong intersectional and regional approach.

3.6 Re-entry post September 11th

By mid November 2001 with the gradual fall of the Taliban re-entry to some areas was possible. However most staff that had been on stand-by were sent home. The team going back into Afghanistan were almost completely new. Furthermore, a great deal of written information (reports, memos, data) were scattered, lost or destroyed. There was very little hand-over to new staff and a great loss of institutional memory and experience from the programme. Headquarters staff provided the main points of institutional continuity. In December MSF re-established its presence in Kandahar. The approach taken on re-entry was first to do a clean up, carry out a stock-take operation and then get programmes back on track. In Herat and Badghis this was easier as the evacuation and events in Afghanistan had not dramatically affected them. With the new improved freedom of movement (mainly in the west) it was possible to access previously inaccessible areas17. The approach taken during this period was to phase out completely from PHC activities in Herat, and to continue working in the IDP camps for as long as they existed. The PW support was also to come to an end, the IDW to phase out. The EWS was to be re-established and prepared for phase out. The emphasis was to expand activities in the remote areas

17 Progress report period from 1st January 2002 till 30th April 2002.

14 MSF Holland Afghanistan Evaluation Internal use only especially in the less accessible areas of Badghis. As an initial step support was given to the district referral hospital in Badghis (QIN) and BHUs were established. As the Herat IDPs began to return home MSF’s medical role in Maslack and Shaidai was to decline18. The Balamorgab clinic was handed over to the MoPH. The BFD in Badghis, which had been planed for 2001, was eventually completed in 2002. In Herat it was never actually undertaken. In Kandahar none of the EDR activities were reactivated. The emphasis here too was to be on the remote areas and activation of ‘safe motherhood’ programmes. An exploratory mission was undertaken to determine location and feasibility of such programmes.

3.7 Monitoring Repatriation

MSF re-focussed attention on monitoring the repatriation process as refugees from neighbouring Iran and Pakistan began returning in view of the changing political landscape. To this end MSFH became involved in health activities and information gathering in the transit centre in Kandahar for returnees from Pakistan and in the transit sites dealing with returnees from Iran. Throughout the first half of 2002, MSF also continued working in the Spin Boldack and Chaman camps providing a mixture of health and nutrition services as well as water and sanitation activities and collecting data for advocacy purposes.

18 CMT Report Jan-Apr2002

15 MSF Holland Afghanistan Evaluation Internal use only

4. Programmes Ongoing at start of Period evaluated (May 2000)

4.1 Western Districts Primary Heal Care programme (PHC)

The western-based projects supported a village-based health system in collaboration with local partners, with an emphasis on maternal and child health care. Emergency preparedness was also supported through training by MSF expatriate doctors and pre-positioning of emergency stocks. This was a comprehensive health programme but with a focus on MCH, i.e. women were the primary targets of the programme.

The project purpose was to reduce morbidity and mortality in the rural districts of Western Afghanistan – those targeted through a preventative community based health approach.

The period evaluated saw the third phase (and final year) of the Western Districts Health project. In Phase III, the local NGO partner CHA had overall responsibility for daily implementation and management, with MSF providing technical, logistical and managerial back up, as well as funding support. Phase I of the project had been started in 1998 after an extensive health needs assessment undertaken by MSF and the national NGO partner CHA. There was a mid term review after the first two years of the programme which led to recommendations for the third phase. The mid term review concluded that the joint project of MSF/CHA was coherent with the respective policies of both organisations: ‘The decision to strengthen the existing health structure, dealing with curative care, safe delivery for women, vaccine-preventable diseases and increased knowledge about health and personal hygiene is an appropriate response to the needs of the population19’. At the end of the three-year project period the program was to continue to be managed by CHA until such a time as there is an MoPH presence that can effectively manage the program itself.

The design and content of the project was based on the assessed health needs of the population, the prior experiences of MSF and CHA in implementing other primary health care projects in Afghanistan, and the experience of both organisations in working in Western Afghanistan under the Taliban administration.

The project rationale included:20 • 20 years of conflict had resulted in devastating consequences for infrastructure and resources for health and education. • appalling health statistics particularly for women and children • the imposition of a harsh form of Islam by the Taliban leading to decreased access to health for women • the target area - Ghurian, Zendajan and Kohsan, the most westerly districts of Herat province and the base of the Mujahedeen commander Ismail Khan, was severely affected • the proximity to the Iranian border made it the home of thousands of refugees who had been returning slowly from Iran • the outcome of the survey results in Kohsan indicated very poor health capacities and services in the district, while being a potential area for returning refugees from Iran • through its presence and capacity in the three targeted districts MSF/CHA would be able to monitor, assess and respond to public health emergencies, including malnutrition, through the proposed project • women could be employed and thus receive ongoing education and further training thus providing a legitimate means to gather together and share their experiences • a partnership between the organisations offered the best opportunity to provide an appropriate, cost efficient and sustainable project • the desire by MSF to get out of the city where it was difficult to operate because of daily interference of the Taliban21

At the time of the evaluation the Western districts project had finished. During the latter half of 2000 the office building for the third phase of the Western district PHC program, Kohsan, was completed

19 Mid Term Review MSF-H/CHA, IN GHURIAN and Zandahjan Districts, February 2000 20 Project Proposal PHC project Western Districts Heart Afghanistan, Phase III July 2000 to June 2001 21 Oliver Matthieson - Country manager from January 1999-June 2000

16 MSF Holland Afghanistan Evaluation Internal use only and the clinic rehabilitated. During the first quarter of 2001 the PHC was functioning well with full necessary staff, drugs and supply of medical materials maintained. The training of the TBA continued. By July 2001 the Project was implemented through CHA, with MSF as umbrella donor, providing supervision and capacity building. This situation continued until March 2002, after which time MSF discontinued its support leaving CHA to find a donor for the programme. The contract with CHA had expired in June 2001 but financial and drug support had continued for another nine months. The Project had no current donor the time of this evaluation field visit. The project sites were not visited.

The mid term review found effectiveness was harder to judge as the objectives of the project were, though specific and relevant, neither measurable (quantitatively or qualitatively) nor time bound and some were not achievable19. Comments cannot be made on the effectiveness of the programme since the mid term review, as documents provided to the consultants were limited to the proposal and four monthly CMT reports. Current project staff had limited knowledge of the programme as it had operated before their time and the project site could not be visited due to time limitations. National project staff familiar with the project were interviewed as well as the CHA Medical Co-ordinator22 who was very happy with the MSF support and felt it gave both agencies an opportunity for learning. However he felt there was some confusion over whose responsibility it was to find an institutional donor. CHA felt they required more support from MSF with this. In fact the proposal states that during phase III MSF and CHA would contact institutional donors to find independent funding to continue the program. In this regard MSF did give some assistance in contacting passing proposals on to donors however CHA had expected more assistance. Dr Mohammed, the CHN Medical Co-ordinator, was clearly committed to the clinics which they had been supporting with MSF. He was so concerned about what was going to happen the clinics he said he had become obsessed and could think about nothing else “where ever I am, what ever I am doing, how to get support for these activities is on my mind”, he said. He felt that, along with MSF, CHA had done so much that he did not want to see collapse. The programmes were beginning to suffer as a result of MSF no longer supporting them (since March 2002). The supply line for some drugs had ruptured. So far however (June 2002) they were largely maintained by the revolving fund.

From MSF's point of view the programme was closed and everything went fine; they also received back one vehicle. However the effect of the withdrawal of MSF on the programme was not formally monitored. ‘We heard some “rumours” that the situation in the clinics we supported in the last four years is not really exceptional and probably are in need of more support from International organisations’23.

4.1.1 Conclusion

This seems to have been an appropriate programme given the health context focusing on mother and child health needs, as well as, the developing the capacity of the local NGO. An external evaluation of the three-year partnership between MSF and CHA was proposed from which it was hoped to gain lessons learned from a partnership approach to humanitarian assistance. It does not appear that this evaluation was undertaken.

This partnership model seems to have provided MSF with the opportunity to contribute significantly to the development of comprehensive health services that targeted women in a culturally appropriate way. The fact that CHA had a long-term commitment gave MSF a partner to hand over to from the start. However more support could have been given to help CHA secure another donor.

There were problems with this project however, as it did not fit with MSF's view of its role as an emergency organisation responding to health needs arising out of ‘crisis’. MSF was not comfortable with the donor like role it began to assume. It was viewed by some as not fitting in with MSFs core competencies.

This is an important model of approach to intervention in Afghanistan as it allows for MSF to respond to long term problems by teaming up with those who have a more long-term commitment. However it does seem that MSFs role became more donor-like and not ‘hands on’ – this could be changed in the future if the organisation were to become involved in a partnership again.

22 Dr.Mohammad Fareed Waqfii (CHA Medical Co-ordinator) 23 CMT 4 monthly report Jan-Apr 2002

17 MSF Holland Afghanistan Evaluation Internal use only

In is not clear why the PHC systems established in the three districts were not integrated more into MSF's Emergency Drought Response and in the establishment of a referral system linked with the paediatric ward activities. Links were maintained with emergency preparedness and the provision of resources for outbreaks but this could have been built on by MSF allowing for the co-ordination of an Early Warning System similar to that in Kandahar.

18 MSF Holland Afghanistan Evaluation Internal use only

4.2 Emergency Preparedness and Response (EPR) Kandahar

The EPR commenced in November 1999 in Kandahar Province, as well as Tarinkot and Derawod districts in Oruzgan, South West Afghanistan. The programme was initiated after a cholera outbreak in the city to which MSF responded. Previous to this MSF was supporting PHC in two districts in Oruzgan province just North of Kandahar Province. There had been a number of outbreaks of cholera in Oruzgan and in general infectious disease was seen as a serious public health problem in the area. Other influencing factors contributing to MSF's decision to become involved in the Regional Hospital were: • the change of strategy, to initiate a presence in the cities, by the OD • pressure from the authorities • a desire to be ‘hands on’ and visible

4.2.1 Early Warning System

The project purpose was to improve case finding, reporting and management of infectious diseases, and co-ordination of epidemic control strategies24.

This was to be achieved through a reduction in case fatality rates and cross infection in the Infectious Disease Ward of the Mirwais Hospital, as well as by setting up an early warning system in 17 sentinel sites (clinics run by National NGOs) for early detection management and containment of diseases with epidemic potential.

The Kandahar EWS was ‘light’ on MSF resources as it operated by through existing systems. A national MSF staff doctor was responsible for MSF's involvement at the time of evaluation. In the past the project had relied on one national doctor and one expatriate.

The programme aimed to reduce the threat from epidemic diseases, initiate prompt response and increase local NGOs capacity to respond promptly. Though the programmes statistics and reports were not available on site (the office had been looted by the Taliban after the MSF evacuation), it is the evaluator's impression that the programme was largely successful in achieving these objectives. Through the recall of the national doctor an idea of the epidemic warnings responded to in 2000 and 2001 could be reconstructed25.

Table 2: Outbreaks responded to by MSF in 2000 and first half of 2001, Kandahar Province Disease no. of ‘epidemic warnings no. of ‘epidemic warnings responded to’ 2000 responded to’ 2001 (Jan-June) Measles 12 13 Suspected Cholera 34 5 Meningitis 0 1 Pertussis 1 3 B.Diarrhoea 1 Typhoid 1 Malaria 1 Total 50 22

The EWS was appropriate to the health needs of the population utilising and building capacity in already existing structures. It also utilised MSF medical expertise and was emergency focused. It allowed for advocacy out of medical action. Dr Naem Rahimie (AHDS) recounted an example where when during a measles outbreak in Tarinkot, AHDS was having difficulties getting vaccines from UNICEF so MSF contacted UNICEF NY after which AHDS got the vaccines immediately. The response to warnings of epidemics included a site visit and assessment, case management, determining the source of contamination (e.g. water point), containment if necessary through vaccination and education and other preventative measures. MSF (in collaboration with REMT, UNICEF and WHO) provided vaccines, health education and where needed, drugs, chlorinating kit for wells, Chloramphenicol and Co-trimoxazole.

24 Emergency Preparedness and Response, Project Proposal, Oliver Mathieson 25 Dr Kabirullah EWS Kandahar

19 MSF Holland Afghanistan Evaluation Internal use only

The approach MSF took in ensuring a co-ordinated response meant close collaboration with National Partner NGOs (IbnSina AHDS), MoPH, WHO and UNICEF. MSF were continuously mindful not to create dependency by replacing other organisations whose mandate it was to provide components of a required response. The project was ready to be handed over to partners prior to Sept 11th however post evacuation there were many changes in staffing at the clinics so more work had to be done to bring it back to the standard previously achieved. The programme was therefor extended to allow for this.

Discussions with local partners and project staff revealed good co-ordination and co-operation on this project26. The continuity of the project has been maintained through an MSF national supervisor.

4.2.2 Infectious Disease Ward (IDW)

MSF's involvement in the Infection Disease Ward followed the cholera outbreak after which MSF rehabilitated the IDW. The IDW also served as an important part of the EWS. Infectious diseases were and are a significant problem in Afghanistan however, effecting appropriate management of diseases through hospital structures has not been without its difficulties, e.g. • late presentation of patients • difficulties in influencing change in medical practice.27

The strategy of supporting hospital wards on a short term basis also raises questions about • absence of an appropriate partner for hand over • increasing ward capacity with no long term plan (so drawing on resources and services)

However MSF were ‘happier’ with the ward's performance than that of the paediatric ward in Herat. The IDW was rehabilitated and extended. It was clean and well run. There was a huge contrast with other MoPH wards. Treatment protocols were developed and used, and a good standard of nursing care was evident. The employment of an MSF national nurse manager contributed to the successes in ward management. The exit was being treated in a staged approach. The project had been extended until the end of 2002 and at the time of the field visit (June 2002), staff incentives had been reduced. However the plan to exit the ward was not welcomed by the MoPH Hospital Director, ward doctors or national MSF staff who felt that the health conditions had not changed and the drought continued. The general impression was that and if MSF left without a hand over partner it would be ‘a disaster’, and that everything would collapse. The ward had a very good reputation among the community28.

For the first 6 months in 2002 of the IDW the reported statistics showed the following:29 OPD saw 4299 patients and admitted 1544 129 had self-discharged 69 deaths (these were in the ward >24hrs) 83 deaths (present in the ward <24hrs).

Case fatalities were reported as follows No. PMR%30CFR%31 Acute viral hepatitis 8 5 23 Acute watery diarrhoea 34 22 5 Cholera 000 Amoebic Dysentery 13 9 8 Dysentery Bacterial 5 3 10 Malaria PF 000 Malaria PV 000 Measles 0 0 0

26 Dr Naem Rahimie (Project Co-ordinator AHDS), Dr M. Daud Zahir (SW regional director, IbnSina), Dr Bawar, Dr, Bawar, Rohangi AHDS Clinic. Dr Popol (WHO). 27 While the expatriate doctors were present practice would change but revert very quickly when the left 28 “The reputation of the IDW extends as far as Pakistan” Dr Zakaria (ward doctor) “one month after MSF leave 3 years will be lost, This is the only ward that admits treats and discharges” 29 01/1/02 –30/6/02 IDW mortality and morbidity statistics 30 Percentage Mortality Rate 31 Case Fatality Rate

20 MSF Holland Afghanistan Evaluation Internal use only

Meningitis 22 14 17 TB 118 Typhoid 12 8 7 Others 57 38 21 Total 152 100 Where international guidelines were available for Case Fatality Rates these were compared as follows32:

Acute Viral Hepatitis: Symptomatic management, no actual treatment, (assuming hepatitis A, expected CFR is - 0.1% but higher in children under 5 - 0.15% and the elderly - 2.7%) (assuming hepatitis B, 1% of those hospitalised in the acute phase would be expected to die). Therefore the rates here would appear very high but it is possible that there are diagnostic errors; the patients presenting with chronic complications of the disease or the symptoms being manifestations of other diseases/conditions. The case fatality rates presented do indicate that the cause of death requires further investigation.

Acute watery diarrhoea It is assumed that cases were viral gastro-enteritis (as all except one of the deaths occurred in children less than 5 years old). There are no CFRs provided but using cholera as a measure 1% to 2% of cases will die with appropriate treatment. On this basis the CFR is a little higher than expected.

Dysentery Bacterial The CFR among hospital cases was 20%. Reported CFR therefore was half what was expected.

Meningitis Assuming this is meningococcal meningitis, 50% can be expected to die if there is no treatment. The CFR is 5% to 15% with good treatment in which case this outcome would seem adequate.

Typhoid If no treatment 10% could be expected to die for all cases including OPD. Prompt and correct management would achieve a mortality of <1%, Therefor the CFR was higher than expected

Target case fatality rates were specified for cholera, measles and malaria in the project proposal but there were only a few reported cases and no deaths. The comparison with international guidelines shows variable success and there are indications that causes of death may need to be further investigated. Also mortality is significantly affected by late presentation (over half of the deaths were in the ward for <24hrs when they died).

4.2.3 Conclusion • The EWS was an appropriate and effective strategy, which was implemented well under the supervision of a national MSF doctor. • The IDW was an important part of the EPR and was managed well. Anecdotally the level of service and quality had improved dramatically through MSF involvement. A reduction of Case Fatalities cannot be adequately judged through this evaluation and there are indications that the causes of deaths need further investigation. • MSF involvement in the IDW was influenced by what the Taliban would and would not permit as well as an analysis that cities were in greater need33. • By the end of the period under evaluation MSF were planing to withdraw from the IDW in a phased manner . There was recognition that MSFs withdrawal from the IDW would most likely be in the context of a need for continued external support but without an appropriate hand over partner. There were differing views as to the extent to which achievements would be reversed within this context. 34

32 Ed. Benenson, Abram S. The 16th ed. 1995 American public health association 33 MSF had wanted to become involved in MCH coordination in Kandahar and prepared a proposal to this effect however the Taliban would not allow it. (Personal comms:Fried Anepool & Helen O’Neill) 34 The prominent view amongst those interviewed during the evaluation was that the MSF plan for withdrawal was premature as MOPH was not yet in a position to take over the running of the ward from MSF Yacoub MSF Nurse, Dr. Popol WHO, Mohamed Osman Hospital Director, Dr Zakaria MOPH, Dr Jabar MOPH.

21 MSF Holland Afghanistan Evaluation Internal use only

4.3 Bala Morgab, Badghis, PHC

The programme in Badghis was a comprehensive health programme with a focus on MCH.35 The project purpose was the institution of a community-based primary health care system with emphasis on maternal and child health care, primary health care and emergency response 36. This evaluation saw phases two and three of the project with the third phase concentrating on handing over to an appropriate INGO.

In 1999 as a result of improved security, a large portion of the war-displaced population of Shaidai IDP camp returned to their areas of origin. MSF ceased its activities in Shaidai camp in March 1999. In April 1999 MSF started to establish health services in Bala Morgab where the population was returning in the absence of health care facilities. After extensive renovations and de-mining of the premises, MSF opened its BHC/MCH clinic in September 1999.

Rationale for commencing the project included37: • there had been no EPI activity since 1977 • MSF's desire to follow this population, that it had engaged with in Shaidai, back to their home town in order to provide basic health services to all returned IDP’s as well as those who did not flee • MSF's experience and capacity to set up basic health care where none exist.

The comprehensive PHC project in Bala Morgab was seen by MSF as being very successful in reaching the community with preventative and curative care38. Prior to Sept 11th it was being prepared for hand over to an agency better able to implement a more developmental approach, and incorporating non-medical activities. A final evaluation, incorporating lessons learned, was recommended but there is no evidence this was done. In early 2002 the MoPH was determined to take over the clinic and did so. MSF guaranteed a six months provision of drugs, and two months’ salaries to the staff. The clinic’s medical staff were quite unhappy with MSF's decision to hand over to MoPH and not to another INGO, as salaries were the same39. The clinic gave MSF the base from which to incorporate part of the EDR, namely Blanket Feeding, Supplementary Feeding and Measles vaccination of all children 6mths to years in the Morgab district. MSF also responded to a cholera outbreak achieving a CFR <1% .

4.3.1 Conclusions

• the comprehensive PHC programme was appropriate to health needs and MSF core competencies • the strategy was not in keeping with the analysis that cities were worst affected by the conflict. • Although part of the rationale for establishing health services in Badghis provinces fit with MSFs desire to continue its commitment to a population with whom it had already engaged, bringing

35 Personal communications Oliver Matthieson - Country manager from January 1999-June 2000 36 Project proposal Phase 2 PHC Bala Morgab Badghis. 37 PHC Murgab district, Badghis Province, Afghanistan, Phase 2 1st April- 31st March 2001 38 PHD trip report by Christa hook April 2001 39 CMT 4montly report 2002 Jan-April

22 MSF Holland Afghanistan Evaluation Internal use only

health care activities to where there had not been any may conflict with MSF policy to respond to health needs arising out of the crisis rather than ‘normal’ health needs40.

40 “Most of the rural areas have not been affected directly by the conflict. They never got the chance to get developed over the last 20 years. But the fact remains that Primary Health Care does not exist but never has existed. Building up a system that even never has been in place is certainly not feasible for an INGO and for sure not by MSF. It is long term and very developmental. MSF does not have the expertise nor the resources to do it, and it results in a lot of frustration on the ground expats as well as local partners.” (OD trip report 1999). Also see MTP page 13 ‘ MSF essentially performs its work in periods of crisis- when a situation is no longer held in the balance and the very survival of a population may be threatened’ …. A crisis is different in time scale and severity from normal poverty inequity or exclusion

23 MSF Holland Afghanistan Evaluation Internal use only

5. Programmes started in 2000-2001

5.1 Emergency Drought Response

MSF's Emergency Drought Response (EDR) was developed as a result of a rapid qualitative assessment conducted by MSF during July and August 2000 in the three drought affected areas of Afghanistan in which it was operating. The assessment confirmed the FAO/WFP findings that in most areas assessed, food insecurity was at the stage of ‘crisis in food’.41

MSF felt the most appropriate response to the crisis was the implementation of a General Food Distributions (GFD) for drought affected areas. However, as MSF is a health agency and not a food agency it saw its role as advocating for this response.

There has been an ongoing debate within MSF regarding its potential role as an agency which implements GFDs. This debate took place throughout the 1990s and involved other MSF sections. MSF Holland adopted a policy of not implementing general rations for a variety of reasons, e.g., programmes would detract from health programmes, lack of logistical capacity and expertise. This position was re-visited most recently at a medical co-ordinator day in Amsterdam (2001) and the same conclusion reached. However, MSF staff were nonetheless conflicted about this position in Afghanistan as the food crises was clear and there was a recognised lack of WFP Implementing Partners (IPs). MSF’s decision not to act as an implementing partner was based on lack of capacity and competence in this area and the potential political difficulties of establishing and monitoring food distribution systems as well as the perceived encroachment on its independence.

The stated Project Purpose of the EDR was to reduced the level of malnutrition among children <5 and detection of trends in malnutrition and potential disease outbreaks. Programme components were to: 1. incorporate nutritional surveillance activities and enhanced food crisis monitoring in selected vulnerable sites 2. (a) provide SFCs in selected communities and districts in Herat, Badghis and Kandahar provinces for the purposes of: • preventing severe malnutrition in targeted children <5 years and pregnant, lactating mothers. • being part of an extended nutritional surveillance 3. (b) prevent further deterioration in the nutritional situation and bridge the gap prior to the increased general ration intended to improve health status, through blanket supplementary feeding to children <5 years in identified areas to(this component was added in May 2001) 4. prevent mortality due to measles outbreak through mass measles campaign for children between 6 months and 15 years

5.1.1 Incorporated nutritional surveillance

Initially the seventeen sentinel site clinics were to be the bases for monitoring trends in the nutrition status of attending children. This included screening children by MUAC with malnourished children then having weight/height measurement, which would also be carried out at the clinic. This activity was new for clinic staff. The activity was undertaken in all clinics for a few months. However there was a lot of additional work and clinic staff could not see the point of identifying children as malnourished and then not ‘doing anything’42. No extra resources were provided to undertake the nutritional surveillance. AHDS suggested establishing SFPs in all clinics. This was not accepted by MSF on the basis that a high malnutrition prevalence was not demonstrated by survey42. Eventually it was decided to stop the monitoring in all but three clinics and the IDW. The results of this monitoring were used in a report produced by MSF in August 2001to highlight the deterioration of the situation. All sites showed increasing rates of malnutrition from March 01 to July 01. Though the monitoring of malnutrition in clinics has had biases it can be useful in depicting trends. However to be most useful it would need to be established in normal years (as a baseline) in order to account for seasonal biases. A seasonal increase could be expected during the summer (diarrhoea) months. The GMR in July had

41 Rapid assessment drought report Badghis, Herat Kandahar August 2000 42 AHDS Programme Manager

24 MSF Holland Afghanistan Evaluation Internal use only risen to 30-50%43 in the three clinics (it is not clear whether this was determined by weight/height or MUAC measurement).

After the return of MSF after the evacuation, nutritional surveillance in sentinel sites was not monitored by MSF. However UNICEF commenced supplementary feeding in all of the twelve AHDS clinics with a monitoring component whereby all children were screened and weighed and measured if necessary. This activity was resourced and reported to be working well.

Food Crisis Monitoring

MSF undertook two main surveys on the food security situation during the period of the evaluation. The evaluations collected qualitative information on food security: focusing on how people adapted to insecure food situations in order to withstand expected food shortages. Information was collected on population movements (as an indication of a deterioration in the situation ), community perceptions of drought, assistance provided and the public health situation. Assessments were undertaken in many of the districts in Badghis, Herat and Kandahar and conducted through focus group discussions. These assessments utilised as a framework the evolving MSF methodology on sequential stages of food security/insecurity based on analysing coping strategies and impact indicators. Although there was some confidence within MSF regarding the seriousness of food insecurity (this was in accordance with other agency findings and perspectives) there was also some confusion regarding the lack of supporting evidence of high levels of malnutrition. MSF prudently enlisted the help of Oxfam (who had longer- term food security experience in Afghanistan) in devising and undertaking the assessments.

Teams implementing measles vaccinations and BFD teams were also used to collect food security information. Initially this was informal and ad hoc however it became increasingly systematic and was recorded with MUAC and measles vaccination results in a database. However this database was not available in Afghanistan for review.

It appears that there was much food security information generated but it is not clear how it was used. There were suggestions that field staff lacked capacity to analyse, interpret and present food security and nutritional information in a manner which would have the most impact in terms of advocacy 44. An individual was hired specifically to analyse food security data and “do proper advocacy” However it appears that this role was difficult to fulfil partly because of different expectations of this between the direct line manager and the Medical co-ordinator and HQ45.

After returning to Afghanistan post evacuation the emphasis of MSF programmes had changed and the focus on food security monitoring activities did not appear to be a priority. However in early 2002 a nutritionist was appointed in a shared position between MSFH and MSFB with the main role of monitoring food security. Once again this position experienced difficulties arising out of varying expectations between HQ and field.

MUAC monitoring

As part of the EDR, MUAC assessments were conducted during mass measles vaccination campaigns in the three provinces. MSF measured MUAC on all children from 6 to 59 months old attending for vaccination. Later when the BFD programme began MUAC was measured on children coming for supplements.

As it was felt that weight/height surveys could not be carried out, MUAC was intended to add to the nutritional surveillance information to give a clearer indication of the effects of drought on malnutrition. There had been much internal discussion about the use of MUAC for the assessment of malnutrition. MSF was confident about the indicators value in assessing acute malnutrition (see section malnutrition versus weight/height).

There were biases evident that affected the reliability of the MUAC data • children <1yr old were included,

43 Nutrition Surveillance Afghanistan, MSF August 2001 44 personal communications Fouad Hikmat Afghanistan HOM 2001 45 personal communication: Saskia Van der Kam

25 MSF Holland Afghanistan Evaluation Internal use only

• sick children may have been kept at home during measles vaccinations • it was felt that ill children may have been presented during re-assessment of MUACs 46 • during the re-assessment of MUAC different geographic areas, albeit close to the original areas, were assessed • children coming for vaccination, children presenting for BFD, and children assessed for assessment purposes only, were from different groups and may not be comparable • difficulty in estimating coverage as population figures were not always available

Nevertheless trends were produced by MSF for Daman and Maywand which showed a deterioration.

Table 3 Increasing trends in acute malnutrition as defined by MUAC <12.5cms43 District Date % MUAC<12.5cms Total no. children Daman Feb '01 11.6 6903 May '01 14.1 4420 Maywand Dec '00 11.3 10920 May '01 16.3 857 Aug '01 15.3 12090

An analysis of MUAC measurement assessments undertaken at the start and end of the BFD (figure1) shows the unreliability of MUAC monitoring among the BFD recipient population (figure 2) for either estimating malnutrition prevalence or showing nutritional trends at population level. Numbers of children attending the BFDs were changing over time while different children were attending. It was however assumed (by project staff) that MUAC would show impact of BFDs. As can be seen from the figures below this is clearly not the case.

Figure 3 Comparison of proportions of BFD beneficiaries with MUAC <12.5cms in selected sites at start and end of BFD47

60 1= % MUAC <12.5cms in May 2001 2 = % MUAC <12.5cms in August 50 Najoy 40 Sayeden Pungi 30 Mowmand Sahebzadah 20 Ghulam HaiderIDP Mulah Shakoor 10

0 12

Figure 4. Number of children attending BFDs in selected sites at start and end of programme47

350 Series 1= beneficiary numbers at 300 start in May 2001 250 Series 2 = beneficiary numbers in 200 Series1 August 2001 150 Series2 100 50 0

P

Najoy Pungi Sayeden Mowmand Sahebzadah Mulah Shakoor Ghulam HaiderID

46 personal communications: Lizzy Spengler EDR Nurse Ashgabad 47 Evaluation of the BFD (pilot), Kandahar-province, Afghanistan. May-August 2001

26 MSF Holland Afghanistan Evaluation Internal use only

Project staff felt that MUAC was unable to determine what action was necessary while others questioned its ability to define ‘hot spots’48. Others staff argued that for a more meaningful picture weight/height surveys were necessary.

Conclusions

MUAC screenings, though initially intended to be part of a surveillance to measure trends and complement other qualitative data effectively, began to be used to estimate prevalence of malnutrition in areas and triggered intervention response by MSF. Nutritional surveys properly undertaken are the optimal means of obtaining accurate mortality and malnutrition information. It may be, relatively, a labour and resource intensive activity, especially in open situations, but they can perform a critical role in eliciting appropriate responses and allocation of resources. It is understood that there were many obstacles to the undertaking of wgt/hgt surveys particularly during the period of Taliban control. Culture and climate were cited as reasons why wgt/hgt surveys could not be undertaken . However though impossible to ascertain retrospectively and from the outside the extent to which this was a ‘real’ obstacle. It must be pointed out that other agencies did collect analyse and produce anthropometric (involving weight for height measurements) and mortality surveys (figure 2) which allowed a more rigorous overview and understanding of nutrition situation in those areas. At the same time, restrictions from the Taliban were different in different areas. CDC conducted a wgt/hgt survey in March 2002 in Badghis (an area where MSF were operating), at the same time as MSF were carrying out a rapid MUAC assessment in the next province Kesh-e-Kohna. MSF continue to run a mobile SFC without an indication of prevalence of malnutrition in the area. The use UNISCALE while making adjustments for clothing may have been a viable compromise to address cultural difficulties. Other epidemiological studies of morbidity and mortality determinants could have utilised clinic data and surveys of mortality. The confidence MSF has in MUAC as an indicator undoubtedly influenced its uptake as an ‘easy’ alternative. Yet, at the very least results from MUAC assessments should have been treated with considerably more caution than was the case. Another more important issue relates to the poor practice of employing methods of rapid assessments and monitoring on changing and unrepresentative populations/samples and using the findings for comparison and trend analysis as well as estimates of prevalence.

A surveillance system should be able to identify a problem and elicit an appropriate response. In this case the long-established consensus amongst nutritionists is that MUAC is not the most appropriate anthropometric indicator to use for determining prevalence of malnutrition. It is therefore likely to lack credibility with decision-makers. Data collected as part of a surveillance system should be influenced by how and whether the information is going to be used.

Prior to 2002 there was little evidence or thought given to whether results of MUAC screening were likely to be acted upon by WFP, i.e. the credibility that the surveillance system had.

There was no evidence of technical co-ordination mechanisms prior to Sept 11t. In general there were no nutritional expertise around with other organisations in the MSF project areas49. MSF did collaborate with Oxfam on food security assessment. Immediately post Sept11th MSF did not appear to have been actively engaged with developing nutritional technical co-ordination 50. It was only after September 11th that the country became ‘flooded’ with nutritionists. These nutritionists had almost daily meetings in Islamabad and were establishing methodologies and procedures for working. MSF had no office in Islamabad to start with and the field was somewhat wary of accepting dictates from agencies and their staff that they had not seen or worked with before 51“ Having returned following the evacuation MSF did not appear to be actively engaged with the establishment of a nutritional

48 Assessment Kesh e Kona, Rik Nagelkerke 49 Personal Communication: Saskia van der Kam 50 Re period post Sept 11th ‘MSF were not interested in co-ordination’ Annalies Borrel, Nutrition MoPH (seconded from UNICEF)

51 Personal Communication: Saskia van der Kam

27 MSF Holland Afghanistan Evaluation Internal use only surveillance system which was evolving in the country52. Co-ordination was viewed with suspicion by some MSF project staff due to fears that it might encroach on independence53. Inter-agency technical co-ordination should be actively encouraged within MSF.

MSF developed a capacity to conduct food security assessments, which allowed them to better understand the nature of the food crisis. This capacity should not be lost through changing teams and priorities but should be institutionalised in a country.

MSF as an institution needs to develop an increasing competence in integrating food security, health status and nutritional status analysis and be able to support these in the field. In particular, MSF’s expertise and health competence gives it a comparative advantage in analysing the role of health factors in nutritional surveillance or surveys.

5.1.2 SFCs in selected communities

This evaluation will not review the debates about the utility of SFP as a strategy to reduce under five malnutrition in the absence of a GFD but rather will examine the successes and failures of the SFP against stated objectives and against MSF policies and, where possible, performance indicators.

The implementation of SFCs is usually indicated with a high malnutrition prevalence (i.e. >10% with aggravating factors54. In Afghanistan the decision to establish SFC was part of the EDR proposal to prevent severe malnutrition in <5 year olds and to be part of an extended nutritional surveillance55

MSF established three Supplementary Feeding Programmes in Open Situations These were at Balamorgab, Kishk-e Nakod and Kesh e Kona.

Figure 5 Admissions to Balamorgab SFC56 At Balamorgab the SFC, situated in the grounds of the MSF Clinic, had opened in December 2000 Balamorgab and had grown consistently so that by September 700 2001 there were approximately 1600 beneficiaries. 600 The programme had alarming default rates of 50- 500 Admits<70% 90% and very low recovery rates (5-40%) 400 admits70-80% 300 indicating that, by MSF standards, the quality, Total accessibility and acceptability of the programme 200 was poor. Project staff pointed out, however, that 100 the exits were low in comparison to the 0

Jul-01 admissions which could have resulted from a very Jan-01 Mar-01 May-01 Sep-01 Nov-01 Jan-02 Mar-02 May-02 poor food security situation. The poor recovery could have meant that children were not gaining weight as the supplement given was actually replacing food rather than supplementing it. As such the programme was acting as a holding operation rather than treating malnutrition. Length of stay for those discharged in January was 142 days. In addition the programme, being a dry SFP, may well have been subject to the abuse which occurs when admissions and food distributions are done daily – the same children being admitted to the programme several times (as occurring in Maslakh camp). Reasons for the high default rate57 also include: transient population on the move to the IDP camps, travel distances and times to the distribution site, and cultural factors that restricted female travel. The high number of Kuchi (nomadic) beneficiaries was thought to have affected the high default rate. The programme was adapted to cater for several severely malnourished children for whom there was no TFC.

52 The MSF International Nutritionist attended co-ordination meetings in Kabul but found it difficult to get information from projects to feed into this information sharing forum. Personal communications Elide Marched. 53 Personal communications with Project Co-ordinator , Kandahar. 54 MSF Draft Guidelines 2002 55 Project Proposal EDR Oct 2000 to March2001 56 Figures up to August were taken from Nutrition Surveillance Report, August 2001. Figures for 2002 were taken from monthly reports. Missing data was not available to the consultant. 57 Nutrition Report Afghanistan August 2001

28 MSF Holland Afghanistan Evaluation Internal use only

An evaluation of the SFC58 concluded that the failure of the SFC in Balamorghab was ‘not just associated with the complexities of such a set-up in an open situation’, but also related to the fact that the programme was not designed on the basis of identified need. The programme continued until the end of the period evaluated even though the food security situation had improved and a BFD was undertaken in the area. After Sept 11th the general food distribution and agencies working in the area had increased. Also the drought situation had improved.

Kishk-e Nakod opened in January 2001. The Centre Figure 6 Admissions to Keshk eh Nakhod 59 was reasonably well organised, with a capacity for 1000 admissions (rough estimate of target group: Keshk eh Nakhod 1200). At the end of week 5 a total of 135 children 120 had been admitted, which is low. This led the MSF nutritionist to conclude that either the 100 80 nutritional situation was not very bad, or the Admits <70% malnourished target group was not being 60 Admits 70-80% 60 Total reached . Admissions and numbers treated in the 40 centre did not increase dramatically throughout 20 2001. Statistics for evaluation against reference 0 indicators were not available. After MSF returned Jan- Feb- Mar- Apr- May- Jun- Jul-01 to Kandahar in December 2001, the SFC was not 01 01 01 01 01 01 restarted even though “the staff had saved the materials and were ready to restart”61. It was not clear why this aspect of the EDR was not re- established whereas the SFP in Kush-e-Kohna and Balamorgab were continued through 2002 (at this time the drought in Kandahar continued whereas in Herat there had been rains).

Figure 7 Proportions of Defaulters and An assessment of the Kesh e Kona area was Discharges in Kush e Kona62 undertaken in Sept. 2000 to establish the potential for an SFP63. It revealed that the MoPH in Kesh e Kush-e-kohna Kona clinic fell between Badghis and Herat and 100% that the area was not in fact being supported by 90% either. The only health activities were EPI which 80% 70% was supported by UNICEF. The community had 60% Deaths built a clinic but services were not being provided. 50% Defaulters 40% Discharge The MoPH doctor from the previous year was 30% 20% now working in Kohan (in the CHA/MSF 10% supported clinic). The qualitative assessment 0% confirmed that Keiki was a severely drought Jul-01 affected area with negative health and nutritional Jan-01 Mar-01 May-01 Sep-01 Nov-01 Jan-02 Mar-02 May-02 consequences. It also highlighted important questions about the usefulness of becoming involved in an SFP without a medical referral point, also ‘why (MSF) would go in with a full team and start a SFC if (MSF) wanted more information on the malnutrition of the population….can be done with vaccination campaign and MUAC… in other words extended surveys with the possibility to identify true needs before committing…’64. Other documents show that the measles vaccination and MUAC were undertaken after the establishment of the SFC i.e., in March 200165. The admissions increased up till Sept 11th and then dropped off for the winter months. Closer supervision was given to the centre in 2002 and fluctuations in monitoring statistics sometimes reflect this, e.g. in March 2002 there was a ‘clean-up’ of the register which resulted in a high proportion of beneficiaries being counted as defaulters (>50%). Nevertheless, an analysis of the annual statistics shows that for 2001 there were over 4000 admissions through the year and 2400 successful discharges. The default rate was only 19 %, which is acceptable for this setting. For the

58 01/09/01: Evaluation Score of Bala Morgab SFC: 59 Figures taken from Nutritional Surveillance report, August 2001. 60 Trip Report Saskia Van der Kam Feb 2001 61 Personal Communications, Ismail EDR national counterpart 62 Monthly statistics (data manager Herat) 63 Assessment of Location for Drought response Kush-e-Kohna 64 Assessment of location for drought response, Kush-E –Kohna, Sept.18 2000 65 West Afghanistan, Heart province, Kush-e-Kohna, March9-13,2002

29 MSF Holland Afghanistan Evaluation Internal use only first 5 months of 2002 however discharges were lower, at 63%, and defaulters higher at 35%, which was thought to be the result of closer supervision and consequently a more accurate categorisation of defaulters. There are also more defaulters in winter months as the cold weather discourages movement outside the home66.

It was also reported that the register format did not make it easy to identify defaulters67. The average weight gain was < 3g/kg/day for most of the project duration, which is sub-optimal and the length of stay >60 days for most months. In April the programme changed to become a mobile service after a MUAC assessment68 showed 14% malnutrition (no change from the previous year) and that services were not reaching people in remote areas due to distances that needed to be travelled to receive supplements. In spite of the fact that having screened 2330 children in 3 villages one month later using MUAC and identifying a GMR of only 4% and a SMR of 0,3% the mobile programme was never the less deemed necessary69.

Figure 8 Kesh e Kona Admissions

Kush-e-Kohna

500 450 400 350 Admit<70% 300 admit70-80% 250 readmit 200 150 Admit total 100 50 0

Jul-01 Jan-01 Mar-01 May-01 Sep-01 Nov-01 Jan-02 Mar-02 May-02

A nutrition programme that targets at children at risk of becoming malnourished in a situation of drought, food insecurity and poor feeding practices with supplements is not inappropriate. These ‘safety net’ programmes have been established in the past and have been linked with nutritional screening in clinics in areas prone to drought and food insecurity and child malnutrition. However MSF guidelines suggest that SFPs are appropriate in the context of a high malnutrition level.70An appropriate assessment71 of the prevalence of malnutrition had not been done prior the decision to establish centres.

Given that the acute malnutrition rate rarely exceeded 10%(Wgt/hgt) in any of the surveys conducted over the period in question and that there were other evident health needs the focus on SFP may not have made the best use of MSF resources.

In the absence of reliable data on the nutrition situation the provision of appropriate complementary foods to moderately malnourished under 30 month olds, (who it appears were the most at most risk of acute malnutrition72), through expanded clinic activity may have been a more appropriate strategy. This would act as a buffer against the effects of drought and prevent those individuals from becoming severely malnourished (UNICEF did this after Sept 11th in the AHDS clinics with whom MSF were working in Kandahar). There is a subtle but important difference between this and the response

66 MSF nurse for KeK SFC 67 Personal communications K-e-K SFP Nurse 68 West Afghanistan, Herat province Kush-e-Kuna district Explo mission report MSF March 9-13 2002. 69 MSFH Western and Southern Afghanistan, April 2002 Report 70 GMR >10 or 15% Table 3.2 MSF Guidelines 2002 71 this refers to the use of appropriate sampling techniques and the use of appropriate indicators. 72 In all surveys younger children were more vulnerable to acute malnutrition than older children (Compilation of ACF Surveys 2000), Also SCF US April 2001, Badghis Survey CDC March 2002

30 MSF Holland Afghanistan Evaluation Internal use only established. The programmes would be light and could treat health care and nutrition programmes holistically and would not attempt to effect population level malnutrition which in any case they could not do. There would also be the benefit of strengthening nutrition surveillance as all children entering the clinic could be screened and if necessary weighed and measured. This would be linked to action in that the malnourished children could be enrolled directly in the programme if found to be malnourished. Programmes would be linked to functioning health systems.

SFCs without adequate general food security could not prevent nutritional deterioration amongst the population. That being said it must be acknowledged that the risk of severe malnutrition was likely to have been reduced for thousands of children discharged from Supplementary Feeding Centres. Kesh e Kona had acceptable default and discharge rates for most of 2001 however there were indications that monitoring was not accurate (the staff seemed reluctant to write off defaulters). Statistics were not available for Keshk eh Nakhod but the impression was the same as found all over Afghanistan i.e. the acceptability of this kind of strategy was low due to the factors such as distance, cold, cultural issues around females & travelling, opportunity costs.

Conclusions

• SFPs were not established according to MSF guidelines (as a response to high malnutrition) instead their establishment was heavily influenced by the need to generate data for lobbying73. • SFPs were not established in the presence of adequate food security, which would have contributed to the lengthy duration of stay and high default rates. • SFPs performed poorly according to MSF evaluation indicators. This is related to the fact that establishment was not based on analysis of nutrition situation, general food security was poor and this kind of strategy is not well accepted by communities in Afghanistan. • SFPs where necessary in Afghanistan need to be innovative, e.g. mobile, and include a strategy to help reduce defaulting. A form of contract between the SFC and the family of the malnourished child as part of such a strategy should be considered. • Increases in SFC admission should only be interpreted as an increase in malnutrition after other possible causes for increase have been examined (e.g. same children registering several times)74. • exit strategies should be defined at the start of the project (e.g. assessment of nutrition situation indicating low level of malnutrition, end of drought or reduced numbers attending etc). • the information generated through the centres was sometimes misleading. The statistics were heavily influenced by high default rates and poor coverage (and therefor were not as useful as originally thought). • MSF should strengthen its capacity in Afghanistan for understanding and responding to causes of malnutrition. For this purpose MSF should recruit a field level nutritionist to oversee nutrition programmes, to co-ordinate with other nutrition and food security personnel, to interpret underlying causes of malnutrition, to develop strategies for response and to ensure that nutrition is integrated with MSF's health approach • MSF in Afghanistan need to develop a longer-term strategy for impacting causes of acute and chronic nutrition which seem to be related more to poor feeding practices and disease, exacerbated by the effects of drought. Such a programme might include enrolling malnourished children (less than 30mths) and would include timely treatment of disease, nutrition monitoring and maternal education. Such a programme might include the distribution of appropriate complementary food.

5.1.3 Blanket Feeding Distribution

As the food security situation appeared to decline in early 2001 the EDR adopted a strategy to include blanket feeding of children under five. The BFD aimed to provide a ration of sugar, oil and CSB (approx. 2000 kcals per day) to all under fives. Children were to be measured (using MUAC) on a two weekly basis as part of the ongoing nutritional surveillance.

73 Personal communications with Fouad Hikmat revealed that the EDR proposal was largely an ‘Advocacy proposal’ and that as hard data were scarce the MSF strategy including the implementation of SFCs was to generate data that could be used for advocacy purposes. 74 There was some suggestion from project staff that admissions were higher when expatriates were not present.

31 MSF Holland Afghanistan Evaluation Internal use only

The BFD aimed to75 • supply vulnerable groups (i.e. all children under 5) with a supplement in energy and protein in order to prevent malnutrition in this group • acquire reliable data on MUAC-measurement for all children in the communities where food is distributed in order to detect trends in malnutrition as part of larger surveillance system. • apply the data obtained to give input to the advocacy strategy towards a GFD by WFP

The plan was to conduct a BFD in Herat, Badghis and Kandahar for 8 weeks in selected sites.

The rationale for introducing BFD included: • increasing admissions to SFPs • food crisis indicators becoming more severe • the need to prevent malnutrition • the need to lobby for a GFD

MSF guidelines state that; BFPs aim to provide a complement in case of lack of food due to either insufficient food accessibility or an incomplete food distribution. BFPs are short-term (3-6 months) interventions aimed at preventing deterioration of nutritional status. They are implemented in response to current food crises in large populations.76 They must be accompanied by lobbying for an adequate GFD.

The BFP in Afghanistan was initiated in accordance with the guidelines and included lobbying activities at local, country and HQ level. The advocacy messages appeared to become confused sometimes at local level, focusing on lobbying WFP to provide food for BFD rather than getting WFP to follow up with GFD.

According to MSF guidelines, the closure of a BFP should depends on an improvement of food accessibility and availability, decreased malnutrition, decreased mortality rates, no recent influx of population, and no anticipation of nutritional deterioration.

BFD was carried out in twenty nine villages in three districts of Kandahar Province: Daman, Maywand and Panjwai. It was planned for an 8-week period in each site however some continued for 12 weeks. The strategy was to have the BFD followed up by a GFD from WFP, but there was no agreement with WFP that they would in fact do this. In fact there were indications by WFP that they could not increase capacity (i.e. food was not available) nor could implementing partners be found.

The BFD in most of the 29 villages discontinued without the initiation of GFD and without other indicators for closure present77. The decision to close BFD seems to have been related to MUAC assessment results . There was also disagreement amongst the staff about the purpose of the BFD and the exit strategy of the BFP. The Head of Mission perceived it as primarily serving an advocacy role while others felt it was wrong to discontinue the programme after 8 weeks78.

In the displaced camps however, WFP indicated that they planned to distribute a general ration. However in none of the other sites was there a plan for a GFD at the time of stopping the BFD. The MUAC information obtained from the BFD was not informative. All sites recorded MUACs <12.5cms for >10% of those measured. MUAC data were showing contradictory trends; i.e. rates of malnutrition based on the screening fluctuated widely between fortnightly measurements. Meaningful trends could not be drawn as BFD beneficiaries changed from week to week and numbers attending varied. This undermined confidence in the meaning and utility of the data.

Villagers did not want to travel to other villages to collect their supplement. There were problems with the authorities also, as they did not want MSF going to the villages ‘unescorted’. For this reason the programme stopped for a period of time. Though it was not intended that MUAC would be the determining factor for selection of sites, in fact it did influence site selection strongly.

75 BFD Evaluation August 2001 76 MSF Guidelines 2002 77 Evaluation of BFD (Pilot) May-August 01 78 personal communication National counterpart to EDR nurse

32 MSF Holland Afghanistan Evaluation Internal use only

Another issue was that WFP wanted MSF to allocate wheat and lentils as part of the BFD ration. However, MSF refused the commodities on the basis that it did not want to (and could not) become a WFP implementing partner for general food distribution. In actual fact MSF was an implementing partner for WFP for SFPs and BFPs and had signed agreements stating this.

The BFD in Badghis did not commence prior to September 11th. This was reportedly because sugar and oil stocks were delayed in reaching the sites. However CSB was available from July and the team in Badghis were ready and on stand by. The BFD in Badghis did go ahead after the team returned there in December but the one planed for Herat never took place. MUAC measurements taken during the BFD in December 2001 and Jan 2002 revealed 4.9% of children to be less than 12.5cms. This is compared with rates of around 30% at the time of the measles vaccination in August and September 01. Though a decline in malnutrition may have occurred between August/September and December/ January any decline is unlikely to be as marked as results suggest. Rates are in fact not comparable. Prior to September 11th MUAC was measured on all children aged from 6 months to 5 years who were attending mass measles vaccinations whereas post September 11th during the BFD in Badghis there was a height cut off used of 65cms. The effect of including not using a height or an age cut-off prior to September 11th would have been to inflate prevalence estimates based on a MUAC cut-off of <12.5cms.

Conclusion

• In June 2001 MSF planned to distribute food supplements to approx. 24000 in 3 provinces. Prior to September 11th it had only distributed to approximately 5000 children in one province. • MSF planned the BFD in Afghanistan in accordance with their guidelines • lobbying was included in the BFD strategy however the impact was far less than had been hoped for as the problem for WFP was one of capacity and resource rather than will. Furthermore, the advocacy message became confused. • there were significant time delays in Badghis in initiating the programme, much of this was due to back and forth discussions over type of commodity and efforts to get WFP to supply the commodities MSF wanted. The BFD in Herat was never undertaken. • MSF managed to distribute significant amounts of food in Kandahar and Badghis through BFD (albeit in Badghis 6 months later than planned) • MUAC screenings should not be used to indicate the need for BFD. It might have been better to initiate BFD in a limited defined area (e.g. IDP camps) and continue for longer (as the guidelines suggest) rather than continuing to add more sites for a short period. • Registration of BFD beneficiaries to facilitate monitoring is advised. • where one of the aims of a BFD is to act as a holding operation until GFD can be secured it would be more appropriate not to set arbitrary time limits but to base the phase out on general improvements in food security. Given MSF’s awareness of the constraints facing WFP the adoption of 8-week cut-off points did not seem appropriate. • the strategy to introduce BFD in Morgab district seems logical. The SFC had increasing admissions all the time people were displacing to Herat. In hindsight it may have been better to proceed with the distributing the CSB and not wait for sugar and oil. By the time it actually went ahead general food to the area had increased dramatically. • Whether the BFD in Kandahar prevented malnutrition can not be judged. However given the small number of eventual beneficiaries (5000 beneficiaries in 3 districts with a population of approximately 40000 under fives), the short duration (8-12 weeks) the changing recipient population at each site, the limited success of the lobbying strategy, unreliability of data produced the BFD is unlikely to have significant impact. • Where the appropriate response is to improve food security of populations through general ration provision BFD is a poor alternative. It is acknowledged that MSF do not see it as an alternative but a stop gap solution as such it should be undertaken only if the population can be clearly defined and the end point is linked with an improvement in food security.

5.1.4 Prevent mortality due to measles outbreak through mass measles campaign

In December 2000 MSF commenced its mass measles campaign and succeeded in carrying out vaccinations in all districts in Kandahar as well as selected districts in Badghis and Herat. Measles

33 MSF Holland Afghanistan Evaluation Internal use only outbreaks had been reported in 2000 and 2001 in Kandahar. However in the first part of 2002 none had been reported79. Also numbers of measles cases seen at the clinics in Kandahar dramatically reduced to almost none in 200280. The measles vaccination in this province can be seen as comprehensive and successful.

5.1.5 EDR Conclusion

MSF's policy in Afghanistan on GFD, i.e. that it would not become an WFP implementing partner, made appropriate response to the food crisis difficult. Implementing BFDs as a 'half-way' house in the hope that GFD would be implemented by others (partly on the basis of advocacy information obtained through the BFD), did not prove effective. Implementation of SFPs in open situations where food security was inadequate was almost certain to result in poor programme performance. Without a GFD large numbers of drought affected people had no option but to move to IDP camps where GFDs and other services were available. At the same time health programmes implemented by MSF where general rations were urgently needed were undoubtedly undermined by poor quality and quantity of diets.

Unfortunately, this Afghanistan experience is likely to recur. A similar situation may well arise for MSF in Afghanistan in the near future. MSF has faced the same operational problems in western Sudan. Unless, MSF articulates a clearer and perhaps more pragmatic strategy for dealing with situations where i) food insecurity has worsened, ii)health programmes are being compromised by lack of food iii) general rations are needed and iv) MSF is one of the only agencies in an area, then field staff will continue to be faced with the type of dilemma experienced in Afghanistan. This dilemma involved a strong sense that MSF programmes were to some degree 'missing the point'. At the same time MSF have to be extremely wary (as indeed many other agencies have found to their cost) of becoming involved in general food distributions in situations of structural food deficit where GFDs end up being repeated on a yearly basis as a form of social welfare programme. Oxfam’s experience in Ethiopia over the past ten years provides valuable lessons while MSF Spain’s experience in Mandera, Kenya also shows what can happen for an MSF section with ambivalent policies around GFD in situations of chronic food shortage. The consultants recognise that there is no easy answer to this issue for MSF but do believe that pragmatic strategies can be further developed and that this must take place as a matter of urgency.

79 personal communication Dr Kabirullah and review of clinic data 80 Review of clinic date for the first 6 months of 2002

34 MSF Holland Afghanistan Evaluation Internal use only

5.2 MSF in Internally Displaced Persons (IDPs) Camps

5.2.1 Shaidai

As a result of mass population movement of people from Ghor Badghis and Faryab provinces, mainly due to drought, the camp at Shaidai was re-formed. MSF had previously been involved in the camp from '96-'99 when displacement was due to fighting and in mid October 2000 MSF became involved again in Shaidai camp. Immediately MSF carried out a measles campaign followed by rapid health assessment and setting up a mobile clinic. Cholera was confirmed by the end of week one of the mobile clinic being in operation and a cholera treatment centre was operational within 24 hours.

In general the international community focused its attentions on providing services for the displaced in Herat. However, it failed to provide adequate assistance to the drought affected areas so that the displacement would not happen. ICRC was an exception and took a preventative approach by providing food for communities in Ghor and there is some evidence that this did in fact reduce movement from this area. According to the MTP MSF can become involved in the provision of food, potable water, sanitation and shelter in case of dire need and failure of others to provide these services/resources81. MSF was involved in one district in Badghis from where people were becoming displaced. MSF interpreted the situation as one of dire need but made a decision not to become involved in GFD and so could not help to prevent displacement Involvement in this response once displacement had occurred was appropriate given MSF's mandate and core competencies. MSF also played a significant role in advocating for appropriate protection and services for the internally displaced.

Figure 8. Admissions to Shaidai SFP82 The SFC in Shaidai admitted 1700 children, Shaidai SFC approximately, during 2001. Of exits 84% were 300 successfully discharged and only 13% defaulted 250 from the programme indicating that the 200 programme achieved an acceptable level of 150 performance. However, the average length of 100 stay was longer than expected at 87 days. 50 Overall the Feeding Programme seems to have

0 operated without major problems. Admissions were low, less than 150 per month except for the Jul-01 Dec-00Jan-01Feb-01Mar-01Apr-01May-01Jun-01 Aug-01Sep-01Oct-01Nov-01Dec-01Jan-02Feb-02 first 2 months of the programme and May and June when there were approximately 250 per month.

MSF provided basic health services with MoPH staff. In general the response in the Shaidai camp represents a typical MSF response. The work appears to have been undertaken professionally. Other agencies interviewed agreed.

One issue which was noted was that a significant proportion of feeding centre beneficiaries and clinic attendees appeared to have been from Herat city and areas around the camp itself (in December 2001 54% of children in the SFC came from outside Shaidai camp83). This indicated that a higher level of services were being offered to the displaced population than to the host population. The close proximity of Shaidai to Herat city facilitated people for the city utilising the services. A nutrition survey in February found a prevalence of malnutrition (defined as MUAC <12.5cms) of 3.9%.

5.2.2 Maslakh camp

In late 2000 the numbers of displaced persons increased and Maslakh camp was established. MSF responded by medically screening new arrivals and providing measles vaccination while monitoring the population movement. MSF also provided nutritional support in the camp through supplementary and therapeutic feeding. The first SFC was opened in December 2000. Monthly admissions fluctuated

81 MTP 2000-2002 page 12 (Core Activities) 82 Data from monthly statistics Herat 83 Report on the SFC in Shaidai, Esther Vigneau-Kuisch, Herat, 15 January 2002

35 MSF Holland Afghanistan Evaluation Internal use only but reached over 1000 in August 2001. In December 2001 MSF opened a second SFC which also had a high number of admissions, increasing to 500 for the month of May 2002. Between the two centres this meant there were 1000 admissions in that month. This was quite surprising given that the UNICEF survey conducted in April found a malnutrition rate of 2.9% (w/h<-2 S.D.) which would have meant only about 300 children (given the camp population of 40,000) should have been eligible for admission to the SFP.

The system of providing daily supplements and allowing for daily admissions in two SFCs close to each other meant it was easy to register several times for a supplementary ration. In June 2002 MSF conducted an exercise to see how many actual beneficiaries should be attending , the result was that approximately 500 out of the ‘registered’ 1700 were actual legitimate beneficiaries. It is unclear, therefore, how accurate previous SFC figures are for Maslakh.

Default rates were high and deaths and discharges were much lower than 80%. However, reporting of deaths was encouraged by giving rations for cards turned in. If the programme was seriously affected by significant numbers registering more than once, as would seem to be the case, then this would have undermined the validity of the programme statistics.

Figure 9. Maslakh SFC 184 Figure 10. Maslakh SFC284

1200 700 1000 600 800 500 600 400 400 300 200 200 0 100 00 01 01 01 01 01 01 02 02

Dec- Feb- Apr- Jun- Aug- Oct- Dec- Feb- Apr- 0 Dec-01 Jan-02 Feb-02 M ar-02 Apr-02 M ay-02

New admissions New adm issions Readmissions Readm issions

Figure 11 Admissions to Maslakh TFC84

140 The TFCs had alarming proportion of defaulters as

120 defined by MSF reference values. However due to

100 changes made after investigation there was an

80 improvement in the second 6 months of operation.

60 Recoveries in the form of discharges increased from

40 47% to 65%. Defaulters were reduced from 34% to

20 25%. Transfers to hospital may hide deaths as may

0 high default rates consequently deaths cannot be Jul- Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar- Apr- M viewed in isolation. It should also be noted that there 01 01 01 01 01 01 02 02 02 02 ay- 02 was a change in admission criteria which included the admission of <11cm MUAC and infants <6mths were Total New admissions Readmissions admitted to the infant feeding protocol. These admissions were reported to have increased the length of stay (MUAC being less sensitive to change than wgt/hgt) which potentially could have increased the default rate. The improvement seen in centre performance was in the context of admitting approximately the same number of children monthly. Deaths were occurring very early on in patient's stay prompting MSF to change the first line antibiotic.

84 Statistics entered by Ismail (data manager), Herat office.

36 MSF Holland Afghanistan Evaluation Internal use only

Figure 12.Maslakh TFC July-Dec 200185 Figure 13.Maslakh TFC Jan-May 200286

8% 2% 3% 16% 25%

34% dead deaths defaulter defaulters Discharge discharged transfer transferred to hospital

47% 65%

An evaluation of the centres performance from July to December 2001 revealed a high default rate for the following reasons87

• mothers were reluctant to leave other children and duties to stay at the centre • poor camp security meant that women were afraid to leave homes and possessions • husbands did not want their wives spending time in the centre where there was contact with male staff • fear that while away from the house women may miss being registered for the general ration • women found that their children were not making adequate progress • carers did not want to leave their shelters in the cold.

Surveys

The MUAC survey in January 2002 conducted by MSF found that almost 27% of children were <12.5cms. MSF concluded that children were becoming malnourished in the camp as screening in the transit site found approximately 10% <12.5cms. There were some reports of people entering the transit centre (and being vaccinated and screened) more than once88 and this may have affected comparability. However MSF accumulated a lot of information indicating that the longer people stayed in the camp the more likely they were to be malnourished.

UNICEF in collaboration with CDC89 conducted a survey in April 2002 at which stage the population had reduced dramatically, the food distribution had changed and the weather had improved. Nevertheless the results found that only 11.8% of <5 year olds had a MUAC of <12.5 cms, and 2.9% were acutely malnourished (as defined by % of median weight for height) indicating that MUAC and weight/height did not identify similar malnutrition prevalence in this instance. As in the rest of Afghanistan chronic malnutrition was very high. The entire curve for the survey sample was substantially shifted to the left when compared to that of the NCHS/CDC/WHO reference population, demonstrating that virtually all the children in the survey sample have some degree of chronic malnutrition. The prevalence of acute malnutrition was greatest among children from 12 to 23 months of age. None of the potential risk factors analysed (including the number of years in the camp) demonstrated an association with either acute or chronic malnutrition. Overall, a large proportion of children less than 5 years of age had diarrhoeal disease or acute respiratory infection in the two weeks prior to the survey. The crude mortality rate for children <5 years of age corresponds to rates of 6.14 deaths per 10,000 population per day, 18.6 deaths per 1000 children <5 years of age per month or 224.4 deaths per 1000 children < 5 years of age per year. Deaths were clustered in time, almost half of the

85 Report on the TFC in Maslakh, Esther. Herat, 31 December 2001 86 Derived from statistics obtained from Ismail (data manager), Herat June 2002 87 Report on TFC in Maslakh, Ester Vigneau-Kuisch Humanitarian Affairs Officer MSF Heart/Afghanistan 21/12/01 88 personal communications Eng Ab. Rahman Salahi (Regional Director HRS/Heart) revealed that the MSF vaccination card and stamp were important for camp registration purposes and so those wishing to register more than once would go through medical screening and vaccination more than once 89 Nutrition and Health Survey Maslakh IDP camp Heart Afghanistan, April 2002 . A collaborative study by UNICEF and CDC

37 MSF Holland Afghanistan Evaluation Internal use only deaths happened between September and November 2001. Thirty percent of child deaths were related to pneumonia. It should be noted that mortality related to the 12 months preceding the survey and as such may in part reflect conditions in areas of origin rather than camp conditions as there is no certainty that those surveyed were present in the camp for that duration of time.

However the UNICEF survey concluded that the level of pneumonia and diarrhoea morbidity and mortality in Maslakh camp reflected poor hygienic and shelter conditions in the camp. It went on to say that the high coverage of relief food distribution and the level of acute malnutrition did not justify implementation of blanket supplementary feeding or specialised therapeutic feeding centres. There is some evidence in a camp situation that childcare practices and micronutrients (biologically determined) have great influence on child’s nutritional status. Diarrhoea and acute respiratory infection appeared to be very common in children less than 5 years of age in Maslakh camp during the weeks before survey data collection. Moreover, these illnesses cause the majority of deaths in this age group89. According to the UNICEF survey, measles vaccination coverage was 62% and measles accounted for 4% of deaths in children less than 5 years.

Advocacy

Early in 2002 MSF identified a number of key issues which eroded the rights of the displaced in Maslakh. Among these were:

• inequitable food distribution • lack of adequate protection • insecure environment • inadequate information regarding the return process to places of origin

MSF created awareness about these issues and their concerns with regard to plans for re-registration and return of the displaced. MSF developed a strategy for lobbying on these issues which included raising awareness among agencies at Herat level. A press release regarding the survey findings indicating inequitable food distribution and finally a letter was sent to the UN special envoy, WFP, UNHCR and IOM.

Undoubtedly MSF were successful in raising awareness. UNHCR took the issue of protection and security to Governor Ismail Khan and was reported to have established a clearer protection role in the camp. However UNHCR had significant problems with the way these sensitive issues were handled by MSF. In spite of objections about an IOM planned re-registration it went ahead and a more accurate planning figure for the camp was obtained. There were many suggestions from other agencies (albeit the ones being criticised by MSF) that the way in which MSF ‘speaks out’ is insensitive and not conducive to improving the situation. UNHCR Kabul took this view further by maintaining that the letter written by MDM and MSF and highlighting protection issues would have had the effect of exposing victims and make further investigation of claims difficult or impossible.90 One of the issues highlighted in the letter (information provided by MDM) was later found out to be false, and this was confirmed by MSF.

Some agencies in Herat, including WHO (who were not involved in the camp), criticised the press release about the nutrition survey (based on MUAC) in Maslakh, because of the indicators used. WFP claimed that they were not notified about the survey results prior to public dissemination91.

This evaluation observed that MSF have ‘spoken out’92 on issues of human rights in Maslakh. MSF's independent position on challenging the UN in particular is welcomed by those concerned that the UNAMA has compromised independence for good relations with government93. However, the methods used are not always conducive to finding solutions and have created poor relations with other agencies working in the same situation. MSF are aware of this but, nevertheless, some are prepared to compromise ‘good relations’ for their position on advocacy.

90 Josefa Ojano (Assistant Chief of Mission (Programme) UNHCR OCOM Afghanistan 91 Amir Ibrahim Programme Officer WFP Herat. 92 “By our presence we witness their needs, the causes of their suffering and the failure to assist. We are bound to speak out on behalf of the victims of crisis on these issues” MTP 2000-2002 93 personal communications, UNAMA

38 MSF Holland Afghanistan Evaluation Internal use only

All agencies operating in Maslakh were subject to pressure from the Taliban prior to Sept 11th resulting in an inadequate registration and inequitable food distribution. Agencies tried to counter this manipulation and at times their staff were threatened and beaten while trucks were stoned and robbed94. The level of abuse towards agencies was so bad that the survey team in April had to forgo the last few households as the team were attacked by an angry mob of men and boys89 . This is not an attempt to excuse ineffective humanitarian assistance but rather understand the operating environment.

With regard to Maslakh and the advocacy position the following points are noted: • MSF succeeded in creating awareness on a number of important issues on behalf of the displaced • the weight of advocacy actions found in this evaluation points toward advocacy targets being the UN rather than the local authorities who are ultimately responsible for provision of assistance • some ‘advocacy actions’ were HQ driven rather than field driven95 • the delivery of assistance to the displaced was heavily manipulated by the authorities particularly prior to Sept 11th • MSF strongly criticised other organisations for their failures in a manipulating environment without offering solutions • MSF failed to adequately register SFP beneficiaries yet criticised IOM heavily for failure to carry out appropriate registration in the camp • MSF advocacy actions caused rifts in inter agency relations so that some agencies would be slow to share information with MSF in the future96

Conclusions - Shaidai and Maslakh

• MSF conducted appropriate heath interventions in Shaidai however as with the other SFCs there was no demonstrated need based on malnutrition prevalence for selective feeding in either camp. • Though the February 2002 survey showed virtually no malnutrition (measured by MUAC) the Shaidai SFC did not close as the camp was continuing to register new arrivals. The health facility and the SFP however seemed to be serving a growing numbers from Herat rather than from the camp itself. • UNICEF/CDC survey calls into question the comparability of MUAC with weight/height at population level among this • Shaidai SFP seemed to have operated to MSF standards while Maslakh was open to more abuse and statistics are probably not meaningful • TFC was not accepted well by the communities indicated by high default rates • TFC performance improved dramatically in 2002 with innovative appropriate treatment for infants under 6 months. • changes in monitoring statistics were strongly associated with programmatic changes and changes in the camp e.g. people not coming because of IOM re-registration • Dry SFP in Maslakh was subject to abuse i.e. same children being registered several times • Advocacy actions are thought to be successful in creating awareness and improving protection but not in improving distribution. However they did not demonstrate an understanding of the constraints facing those agencies being targeted by MSF. In some instances this led to reluctance to share information. • MSF should focus first on their own programme effectiveness and provide the personnel to ensure this. • MSF should investigate and analyse the health environment thoroughly and intervene on the basis of findings (however it is recognised that during Taliban rule rigorous assessments were difficult if not impossible). • when operating dry SFPs efforts should be made to guard against gross abuse (e.g. weekly/fortnightly admission/distribution rather than daily, centres should not be in walking distance of one another, etc.) • MSF should test out models of home based therapeutic feeding in situations where inpatient care is not acceptable

94 HRS, IOM, WFP 95 the decision to publish MUAC survey findings identifying inequitable food distribution as cause of increase of malnutrition was HQ rather than field driven. 96 personal communications IOM WFP

39 MSF Holland Afghanistan Evaluation Internal use only

5.3 MSF's involvement in the Paediatric Ward, Herat

Prior to the period covered by the evaluation MSF's strategy was to stay away from cities to avoid Taliban interference97. However with a change of Operations Director came a change of strategy and a decision was taken to support the Paediatric Ward at the Regional Hospital at Herat98. Other factors, which influenced this decision, were: • cities were perceived as more affected by the 'conflict' than rural areas • a city presence would have a greater influence on the Taliban • a continuous presence was seen as desirable in Herat so that "Iran refugees (who) were returning to Herat and (MSF could be) around for temoinage purposes" • the Paediatric Ward gave access to health centres in the town and surrounding districts • a Paediatric Ward would improve access to women • the Taliban would not permit the preferred choice (i.e. the Obstetric Ward) • pressure from the Taliban to provide services in the city • MSF desire to be more ‘visible’ and ‘hands on’99.

The proposal was for a two year period (with an option to continue) commencing in July 2000100. The proposal was based on three main assumptions • security remained stable • the Ministry of Public Health remained stable with good co-operation between MSF and the Ministry • an exit strategy would be developed.

The overall objective was described as reducing morbidity and mortality in children under 14 years of age in Herat province with four specific objectives identified: • rehabilitation of the Paediatric Ward • improvements in medical treatment and care • the establishment of nutritional support within the Ward • improvement in information exchange between the Paediatric Ward as a referral facility and the Public Health Units in its catchment area.

MSF signed MoUs with the MoPH for ward rehabilitation and support. Previously MDM, Pharmacins sans Frontieres and UNICEF had been involved in the ward. For the first year of the project an MSF expatriate doctor and nurse were engaged in the ward intensively and were actively engaged in training, improving protocols, supervision and management.

In MSF's view the second and fourth of the above objectives ('improvements in medical treatment and care' and 'improvement in information exchange between the Paediatric Ward as a referral facility and the Public Health Units in its catchment area' were not met. This evaluation concurs with this view with regard to the latter objective. There is also some evidence that there was limited progress on the other objective ('improvements in medical treatment and care') The assumption that an exit strategy would be developed, which was one on which the proposal depended, was never realised.

At the time of the original assessment there were 44 beds with 60% occupancy. At the time of this evaluation this had increased to 70 beds with well over 100% occupancy (often 2 children to a bed). At the time of the assessment there was a reported 769 admissions in the 6 month period from January to June 1999. There were a total of 2755 admissions in the period December 2001 to May 2002. Though there was no recorded supporting evidence the mortality rate prior to MSF's involvement was estimated at 6% while the average monthly mortality rate over 15months was 9.8% (see figure 13 below). However, this is likely to be a function of late presentation (mortality among patients who were present on the ward more than 24 hrs was 2.5%101). Prior to MSF involvement the default rate was estimated at 30%, while the default rate during MSF involvement has ranged between 13 and 28% (see figure 13

97 Personal communications Oliver Matthieson - Country manager from January 1999-June 2000 98 Trip Report Afghanistan,23/8 - 2/9, 1999, Marcel Van Soest, OD MSFH 99 personal communications Marcel Van Soest 100 Project Proposal Western Afghanistan Paediatrics support (WAPS) 1st July 2000-31th August 2002 101 this breakdown was only available for 2002

40 MSF Holland Afghanistan Evaluation Internal use only below). The disease related admissions were highest for acute respiratory tract infections (ARI) for the winter months and the acute gastro-enteritis (AGE) admissions increased dramatically during the summer months. This is consistent with other’s finding on morbidity patterns (ACF survey). ARI accounted for 40% of all deaths between September 01 and May 02. The mortality rate during the winter months was higher than other months102

Figure14. Discharges, Defaulters and Deaths in the Paediatric Ward as % of exits, Herat103

Discharge,default , death as proportion of exits

120

100

80 Dis charge 60 Self discharge Death 40

20

0

5.3.1 Regional Therapeutic Feeding Centre (RTFC)

Figure15. Defaulters, Discharges, Deaths (RTFC)104

Bar the first month of establishing the separate TFP 120 in the paediatric ward, monthly admissions ranged 100 between 24 and 82. The defaulter rate was very high

80 sometimes 40% (during the period of evacuation this Death Transfers out exceeded 60%). The death rate, except for November 60 Defaulters (10%) was below 4% however this was probably Discharge 40 masked by the transfers out (to the paediatric ward if 20 the condition was very serious which occurred in 15

0 to 40% of cases). The admission and discharge criteria were not exactly the same as the Maslakh Sep-01 Jan-02 Feb-02 Aug-01 Oct-01 Nov-01 Dec-01 Mar-02 Apr-02 May-02 TFP, which admitted under 6month olds and implemented an infant feeding protocol. There was no day to day supervision of the TFP by MSF staff. Compared with standard guidelines this TFC did not meet minimum standards based on poor default and discharge percentages.

The reports for the first year of involvement recognise the difficulties of attempting to change the ingrained practices of the medical staff. However there was signs of slow improvement. With changes of expatriate staff there were changes in perception of progress. In August more doctors and nurses were employed to improve effectiveness. However, after September 11th the medical co-ordinator was so dissatisfied with practices that he recommended immediate withdrawal of MSF support105. The immediate withdrawal did not occur and a more ‘hands off’ strategy was implemented, although it was noted in one report that more supervision not less was required. It was also suggested that “the health department in Amsterdam should develop a sort of continuum in (MSFs) approach, especially when the high turnover affects a project so strongly”106.

102 In some monthly reports Meningitis was noted however others counted TB and TB meningitis instead of meningitis. Deaths discharges and defaulters did not always make up 100% of exits. 103 PW monthly statistics, Herat 104 RTFC monthly statistics, Herat 105 Personal communication with Afghanistan Medical coordinator at Ashgabat Regional meeting. 106 4mthly CMT report Jan-April 2002

41 MSF Holland Afghanistan Evaluation Internal use only

MSF volunteer medics could not “relate programme objectives to their perceptions of MSF as an emergency health organisation”. This probably reflected “the lack of coherent country policy that the volunteers can revisit”.107.

Towards the end of the project it was acknowledged that “no INGO seemed to be interested and the MoPH has not enough resources, however MSF (wanted) to hand over the Paediatric Ward as soon as possible”7. The lack of a viable hand over party did not seem to alter MSFs plan to exit the paediatric ward at the end of the project period (August 2002).

The MTP108 states that MSF may plan to cease actions when:

• the crisis of need has reduced • when others are adequately fulfilling the needs • when MSF can hand over their programmes to a more appropriate partner • when the Population in Danger are able to cope with the situation.

None of these conditions were in place yet MSF still planned to discontinue support. Neither did the project have a natural end.

During the field visit (July 2002) the health needs with respect to the need for paediatric referral services had not changed and though there were more agencies operating in Afghanistan none had been identified to take over support to the PW. The change in MSF priorities and the lack of pressure on MSF to be in the paediatric ward meant that MSF could easily withdraw from the project although, staff interviewed were not altogether comfortable with this strategy. There was a lack of clarity for project staff implementing an ambitious project and a lack of an appropriate withdrawal plan when it came to ceasing support. The initial proposal saw a need for the development of exit strategy. This was not done. The proposal also saw the time commitment of at least 2 years unless there was a plan to extend the time period. There is no such plan.

What is clear however is that the reasons now given to justify withdrawal by MSF109 do not stand up to scrutiny. i.e.

Reasons given for withdrawal • failure to improve medical practice • failure to improve <14 mortality in the province • many more agencies now involved in Herat

Given the level of inputs it is over ambitious to expect medical practice to have been improved in such a short time frame. An improvement in <14 year old mortality in the province would be effected through early detection and treatment and appropriate and timely referral from the districts, however the linkages with the PHC system were not significantly strengthened as staff tended to concentrate on the ward itself. Although there are now more agencies working in Herat there was lack of willingness to support the Paediatric Ward during the time of the evaluator’s visit. Given that MSF planned a commitment to a government structure it would have been more appropriate to intervene in a manner that was connected with the longer-term context. This should have taken a more realistic view of what standards it was possible to achieve and maintain, i.e. awareness of risks of creating a high level of dependency and attempting to raising capacity over and above what could be expected in a ‘normal’ situation (e.g. employing staff over and above normal standards and at rates over and above normal salaries, extending the bed capacity of the ward110). However, this conflicts to some degree with primary MSF policy of providing quality care to best standards possible.

MSFs involvement in the paediatric ward did not arise out of need alone but was a balance between the requirement imposed by the Taliban to be in the city, the need to be seen, the desire to be in the region to monitor the returnee process, the need for a platform that would provide proximity to the Taliban as well as the undoubted need for support to provide a good service for children with serious illness.

107 Progress report last quarter from 01-Jan-2001 till 30-April-2001 108 MSF-H MTP 2000-2002 109 personal communications with project staff 110 MoPH moved out of what was the IDW next to the paediatric ward to allow MSF extent the PW

42 MSF Holland Afghanistan Evaluation Internal use only

However the programme was only justified in the project proposals in terms of health needs. Supporting a central government health structure like the PW is not strictly an emergency intervention and does not have a natural end point. It is difficult to judge confidently whether MSF should ever have undertaken the project. What is clear however is that the project design did not take into account the strategic reasons behind MSFs involvement and level of commitment required for this type of programme. If it had it might have resulted in more achievable objectives and a more obvious end point to the programme.

5.3.2 Conclusions

How MSF should engage in national health infrastructure should be defined e.g. the level of support to be given (above what is considered normal) and conditions for payment and employment of staff within structures when these are above what is considered normal for a country.

Where continuity, management and supervision cannot be maintained by MSF expatriate staff other models should be investigated e.g. national staff supervisors, managers and medical co-ordinators. MSF national staff realising their potential should be seen in terms broader than simply strengthening the channels by which national staff can become expatriate volunteers111. MSF expatriate staff as defined by the MTP112 are expected to

• work alongside the victims of crisis • understand their suffering • reaffirm their humanity • alleviate their suffering • restore their dignity • provide an external perspective on the crisis and serve as a witnesses of the crisis

The short term nature of expatriate staff placements need not undermine this requirement, however the need for effective management of projects and continuity in approach is undermined by short term expatriate contracts and difficulties in filling expatriate positions. In Afghanistan therefore, projects with strong management and supervision components which require continuity of approach (as PW did) should be managed by national staff with MSF expatriate staff fitting in to the structure to work alongside and carry out the other requirements.

The capacity existing in Afghanistan for national staff to ‘run’ projects does not negate the utility of expatriate staff, particularly in respect of the proximity requirement above. Where influence on medical staff practice is an objective, short placements of senior medical professionals for this purpose (not project management) could be considered.

Written guidance by MSFH HQ on appropriate approaches for different health interventions is warranted. Definitions of approach to health interventions, which are part of a national health infrastructure and have no natural end should be made at Amsterdam level and should take into account strategies to minimise dependency and avoidance of undermining existing structures. Management structures can be light and low cost. The criteria for deciding on what programmes to undertake in countries where MSF are working needs to be governed by policies in those countries (where they exist). Continuity in approach cannot and has not been maintained through expatriate staff and this has lead to conflicting and unrealistic programming.

MSF has decided that it will be in Afghanistan for the foreseeable future yet has no policy or guidance which covers how MSF should engage with health infrastructure (formal and informal) in different scenarios (in areas with or without governance.)

The mortality related to ARI particularly during winter is significant which corroborates other findings in Afghanistan. Through reducing risks at a community level early detection and appropriate treatment significant mortality reduction could have been achieved. In the ongoing health emergency in Afghanistan MSF were in a prime position to investigate disease specific mortality and provide an

111 page 21 MTP MSFH 2000-2002 112 page 10 MTP MSFH 2000-2002

43 MSF Holland Afghanistan Evaluation Internal use only overview of the severity of the health crisis which pre dated the drought induced food crisis. However this was not done.

6. Period September to November 2001

6.1 Evacuation

After September 11th 2001 fears of US reprisal targeted at Afghanistan resulted in the evacuation of all expatriates113. MSF was told by the Taliban that it could not ‘guarantee its safety’ which was a clear message for MSF that it was time to leave. By 14th September all expats had left the country.

According to reports114 the evacuation was not well organised. Files were burned as part of the evacuation and electronic data management failed leaving huge information gaps. Kandahar offices and stores were looted, as were all agency offices except for those belonging to ICRC. A car, which could have been taken to the border, was left behind and taken by the Taliban. There was a huge loss of documentation and files when offices were looted, and expatriates left with computers. Programmes could not continue in Kandahar. When National staff left Kandahar some were re-deployed in Pakistan. National staff were under immense pressure following evacuation, they were left with money and communications which, under the Taliban, were hanging offences.

Herat programmes continued operating during the two months that MSF expatriates were out of the country. However no National staff member had an overview of all the programmes, as this had not been required prior to September 11th. National staff showed commitment and capacity to run programmes without expats.

There was almost a complete change over of CMT and project staff on return to Afghanistan. This coupled with lack of an overall overview by the National staff resulted sometimes in a lack of coherence and continuity. After the return there were too many expats on short missions leading to poor continuity and confusion115

6.2 Regional Approach116

MSF-Holland had a presence in Uzbekistan, Turkmenistan, Tajikistan and Pakistan. This regional presence gave MSH a comparative advantage both for keeping abreast of what was happening and for preparing crises scenarios and response. It also allowed an amount of support to be given to the Afghanistan programmes. The following points were noted:

• human resource capacity at regional level and the Turkmenistan CMT were in place to support Herat programme from Ashgabad. MSF's position and contacts in Ashgabad proved beneficial for access to resources (e.g. stocks were mobilised in the country and testing for diphtheria were possible through links with MoH) • MSFH was key in co-ordinating with other MSF sections and in facilitating their emergency response as well as information sharing and issuing joint statements (daily updates and information sharing was very strong). Consolidated press releases were made possible • MSFH was able to assess and investigate access routes into Afghanistan • a statement was made on the food drops and repatriation which was felt to be positive but the question was raised as to whether there would have been an opportunity to have a stronger sectional regional voice • there was good regional level context analysis however the perception from Afghanistan field staff was that there was no overall synthesis and sharing of analysis with staff in the field after the return. On the issue of influx into neighbouring countries the analysis relied heavily on HCR. • MSFH was prepared to intervene on a large scale although in Afghanistan the need was never realised. However there were some costs for other country programmes as a result of the attention given to Afghanistan. Annual planning was delayed and strategic planning was cancelled. The point was made

113 MSF Switzerland expatriates remained in Northern Alliance areas 114 Afghanistan WG in Ashgabad 115 assistant PC Kandahar (Nafai), Marcel Van Soest. 116 Outcome of working group discussions during regional meeting in Ashgabad

44 MSF Holland Afghanistan Evaluation Internal use only that while communications were excellent between sections after Sept 11, information on developments in Afghanistan and operational responses were lacking from other sections after the return.

• There were some views that preparations in Pakistan were, with the benefit of hindsight, ‘an over kill’. Apparently there was huge amounts of unnecessary stock piling with a lot of wastage. A lot of items in Ashgabad and Pakistan were never used and just given away when the crisis did not occur. There was also the view that the EPREP was an over reaction both in terms of stock and people117. • The Tajikistan programme was unable to fulfil emergency orders (EPREP replacement) as no stock was left in the MSF or in world stores. With re-organisation of procurement at HQ level, orders were placed by the end of September although these did not arrive in the country until the end of April 2002. This resulted in operational shortfalls118. • In Afghanistan there was evidence of over stocking in the Kesh e Kona programme with large amount of food at risk of rotting. Wheat and other food stocks were looted in Kandahar. MSF eventually donated some food to the WFP.

6.3 Conclusions

The experiences showed that the capacity for operating programmes and offices existed at National staff level. This capacity should be nurtured. Staffing structures should be revised to envisage a more independent National staff. This does not imply having national MSF counterparts for expatriates. The potential for national (as opposed to expatriate) project managers (PCs, med-cos, etc.) should be reviewed. This would allow for continuity and coherence in the event of evacuation of expatriate staff.

MSF need to invest in improved information management systems. MSFnet.org is extremely valuable. However it is not available to projects and requires constant management. The amount of information displayed on MSFnet.org risks making it unusable. A search system and display of key documents only, in chronological order, would be useful. However, MSF.net.org is not a comprehensive compilation of all programme key documents. Appropriate information systems need to be in place in field offices as well as regular back up systems. Systems can be devised and standardised for all MSF missions. In the meantime there is a lot that can be learned from cross-country information sharing and support within the region.

A debrief of staff on technical and programme level should be in written form and fed into the ongoing lessons learned processes. Debriefs should be undertaken in a short standard format and should highlight issues of concern to Amsterdam e.g. project staff not agreeing with CMT strategy.

The positive experience of good flow of information between the MSF sections should be maintained and intersectional meeting for HOMs and technical co-ordinators should be encouraged, as should regional MSFH meetings for information sharing.

The experience of the evacuation and the period when expatriate staff were absent from Afghanistan demonstrated how fragile a system can be if it is heavily reliant on expatriates. There was local capacity to run effective programmes. Many other agencies with larger programmes rely to a lesser degree on expatriate staff. However the MSF mandate necessitates having expatriate volunteers operating in solidarity with affected communities. The question is whether such extensive expatriate involvement in the management of programmes can undermine National staff capacity which may need to be relied upon in periods like the evacuation.

An internal audit on extent of over stock piling as well as what happened to stock should be considered.

117 Personal communications Andrew Cunningham 118 Tajikistan /Pakistan WG written up by Penny Harrison

45 MSF Holland Afghanistan Evaluation Internal use only

7. Programmes Started 2002

7.1 Emergency support to PHC in Badghis Province

This programme was commenced after an assessment of Qala-i-Nau, Qadis and Jawand districts in December 2001 and January 2002. The programme's overall objective was to reduce morbidity and mortality for the people in Badghis Province including those returning from IDP and refugee camps through provision of health care. This was to be achieved through support to the referral hospital in Q- I-N and the improvement of access to basic health care of the population in remote areas in the districts of Jawand and Qadis. The programme aimed to have a focus on improving safe motherhood.

The rationale for this programme was • the desire by the CMT to extend activities to ‘reach the most remote areas’. • the need for a good referral system for the Province • no other NGOs were keen to move into Q-I-N • the need for safe motherhood strategies

Very quickly the context changed (more NGOs in the area, with interest in supporting the hospital) and at the end of the period evaluated MSF was hoping to be able to hand over the hospital to a long-term development INGO but maintain the MCH clinic/service under MSF. MSF planned to work together with the MoPH on this project and eventually hand over to them.

7.1.1 Conclusion

A review of this programme was not possible as none of the programme staff were interviewed nor was the programme site visited. On the approach to the programme the following points can be made:

• this programme seems to be in line with the Western districts and Balamorgab programme in terms of health activities. • Afghanistan’s appalling health statistics - in particular for women and children, which in part reflect very poor access to primary health care have provided the rationale for implementing some of the MSF programmes. This is appropriate and needs based from a health perspective. However there may be a conflict between the emphasis on bringing services to remote areas where there may have been none before and the MTP which favours needs that arise from the crisis and do not address ‘normal’ health needs which were present prior to the crisis. • MSF were very quick to move into the hospital even though their experience of such interventions had there own difficulties.

46 MSF Holland Afghanistan Evaluation Internal use only

7.2 Repatriation of Returnees

For 2002 UNHCR has planned for the return of 1.2 million Afghans who have been forced to leave their homes to become internally displaced or refugees in Pakistan and Iran. The track records of neighbouring countries, Pakistan and Iran, in forcibly returning refugees are well known and not likely to change, in fact, may deteriorate if it is felt the repatriation is going slow. There are also many concerns among NGOs as to the ‘absorption capacity’ within Afghanistan119. For these reasons MSF felt it necessary situate itself so that it could to monitor the process closely. Its position as an independent NGO meant it was well placed to do this.

In April 2002, MSF became involved in UNHCR transit areas for returning refugees to Afghanistan in Kandahar and Herat. MSF had been denied access to the transit sites120 but was monitoring the repatriation programme since it began in March 2000 through monitoring forms developed by MSF and communication with MDM and UNHCR. In December the programme was suspended.121. Part of MSF's reason for being in Herat was linked to the perceived need to monitor the returnee process122.

The current project provides medical care for returnees and collects information for monitoring. The stated purpose in the draft proposal was to preserve health of Afghan refugees and be an advocate for human rights123. The explicit human rights objective was a new development in comparison with other MSF programmes in Afghanistan. Unfortunately explicit reference to the human rights objective was removed in the final draft.124

The programmes raise questions about MSF’s institutional identity and primary role. This was demonstrated during the regional meeting in Ashgabad where substantially different views emerged over whether MSF was primarily an activist organisation protecting human rights and humanitarian principals or a health organisation responding to those most in need125. The former type of activity does not fit comfortably with the view held by many health staff in MSF that the agency is primarily an emergency medical organisation that responds to important health needs arising out of crisis. These programmes however fit more with the view of some MSF staff that in Afghanistan MSF has an important role as a watchdog on the UN.

In the Returnees Programme both roles conflict the returnees from Iran are not ‘most in need’ in health terms and this raises questions about the use of MSF resources when there are more substantial health needs among the Afghan population.

7.2.1 Conclusions

• MSF as an independent organisation is well placed to monitoring the return process. • this function in a context of reducing humanitarian space is necessary for the protection of human rights. • this project does not concur with views particularly of MSF health professionals and departments that projects should be in response to the ‘most important’ health needs.

119 NGO Co-ordination and some other relevant issues in the context of Afghanistan from an NGO perspective, 9 April 2002. Ed Schenberg van Mierop, ICVA Coordinator. 120 MSF Afghanistan Peshawar Sit.rep 13/3/00-30/4/00 121 4monthly CMT report May –Aug 2000 122 OD Trip Report 1999, Annual Plan 2000, Country Policy 2000 123 Draft proposal Repatriation of Afghan refugees from Iran. 124 Communications with Stefan Savvy HOM post evaluation revealed that he would not agree to a project purpose stating “…and advocating for human rights”. 125 Discussions Regional Meeting Ashgabad

47 MSF Holland Afghanistan Evaluation Internal use only

7.3 Chaman and Spin Boldak camps

Killi Faizo (KF) staging post was established close to Chaman town on the border between Pakistan and Afghanistan in October 2002 to assist Afghan refugees fleeing to Pakistan. MSF rapidly assumed responsibility for measles vaccination, nutritional screening of all children and basic health care in KF.

A survey by MSF of refugees in KF in December found that more than 60% were from Kandahar and 36% from the most northern provinces. The majority were however Pashtun with 22% female households. More than 75% had fled violence although drought, poverty and lack of food were also a factor for many. In the previous 8 weeks a CMR of 8.7 deaths per 10,000 per day had occurred with 94% of mortality due to violence.

UNHCR and the GoP were embroiled in land disputes before finding the locations of Rohgani and Lande Kareze for refugees transiting from Killi Faizo. Subsequently, other refugee camps were established in Chaman, e.g. Dara 1 and Dara 2. MSF began providing BHU support, measles vaccination, MUAC and general health check ups in these camps. They also established a SFC in Rohgani. Following a nutritional survey in January 2002 the SFC was closed. Also, EPR plans for disease epidemic potential was instituted in collaboration with relevant government aid agencies. Health surveillance and reporting were also put in place.

MSF were the first western NGO to get access to IDP camps of Spin Boldak (SB). All the SB camps were set up after Sep 11th. Fifty percent of the IDP population were fleeing drought and bombardment. They predominantly comprised Kutchi although there were some Pashtun fleeing violence. There were approximately 36,000 in the SB camps run mainly by Islamic NGOs. These NGOs had organised and provided food and NFI to a reasonable degree for these camps before MSF became involved. However, health care was not so well catered for until MSF assumed this role. There were five separate camp sites in SB. MSF’s role has involved BHU support, WATSAN activities in some camps and water chlorination in all camps. MSF also provided top up measles immunisation for new arrivals.

Killi Faizo stopped (formally) being a registration centre for new arrivals in March 2002. It then became a transit camp for returning refugees. As of 21st March the BHU in KF began servicing those from the waiting area and those repatriating. As a result those continuing to flee Afghanistan had camped out in a location that has become known as ‘no man's land’ (NML) desiring to be registered as refugees and to be moved to established camps. Population estimates for NML in April 2002 were 20,000 to 45,000 of whom approximately 99% were Pashtun. MSF activities in NML have involved conducting a measles campaign in March, providing a mobile ORS team, water bladder ramp construction and water provision liasing with NGOs on sanitation.

MSF were surprised at the quality of services provided by the Islamic NGOs on entering the SB camps. A project review conducted in May 2002 determined that MSF should stay involved in these camp as advocacy needs were still great, especially for NML. Lack of WFP food rations in SB worried MSF so nutrition was closely monitored through the BHUs. MSF wanted to conduct a nutrition survey in April 2002 but it took a while for a nutritionist to be recruited (only happened in mid June). In May the nutritional situation in NML started to decline so MSF undertook MUAC screening via outreach services. Many malnourished were identified in a few hours so MSF wanted to follow up quickly. Identified malnourished children were subsequently weighed and measured and sent to selective feeding programmes in KF BHU. The SB camp managers (Islamic NGOs) were not active in terms of co-ordination or information sharing. In fact in July IIRO and WAMY only gave three days notice before pulling out of their respective camps.

7.3.1 Advocacy and co-ordination

MSF held frequent meetings with relevant agencies and government staff on the issues mentioned below and also conducted many interviews with refugees, IDPs and returnees. Issues which occupied MSF and were the basis of considerable advocacy and co-ordination efforts regarding KF, SB and Chaman camps and later NML were as follows:

• issues with regard to repatriation of refugees, - monitoring the voluntary nature of repatriation and security as well as food security in places of origin (especially for Kutchi who had lost all livestock)

48 MSF Holland Afghanistan Evaluation Internal use only

• need for HCR to take protection responsibility for returning refugees who become IDPs • whether refugees would be provided with assistance if they chose to go elsewhere other than points of origin. • health issues, e.g. access at point of origin, medical services in transit, • lack of adequate water in Chaman camps • location of Chaman camps too near the border • trafficking to Karachi and elsewhere of young girls for marriage • conflict between refugees and locals over resources and land rights • restriction on freedom of movement of refugees • plans to close S.B camps and move to camp in Kandahar. Uncertainty about when and how and what sort of assistance would the camp offer and by which organisations • lobbying WFP for equitable food distribution in all five S.B camps. Islamic agencies did low profile feeding and WFP only carried out a few poorly spaced distributions. MSF kept an eye on nutrition through the BHUs and also did a MUAC screening • monitoring nutritional status and keeping ACF informed of the situation • in NML refugees were not being registered as refugees and had an uncertain future. Also, humanitarian assistance was inadequate due to political constraints on access and due to refugees boycotting assistance in protest at their situation. . 7.3.2 Conclusions

MSF activities in the Chaman/SB programme were very much in keeping with traditional MSF programmes. MSF therefore fitted in very comfortably with camp health roles in both Chaman and SB

The advocacy and co-ordination activities have been extremely wide-ranging and comprehensive. Although it is difficult to ascertain the impact of advocacy, the information collection and monitoring of the situation in all camps (Chaman, SB, NML and KF) can only have contributed positively to informed decision-making.

MSF were originally planning to close down programmes in the Chaman camps in May 2002 but were still operating in the health and water sector in four of the camps in July.

The main constraint on MSF during the Chaman and SB programme has been human resources which impacted in terms of lack of programme continuity, e.g. not able to get a nutritionist. Also, there was still CM or medical co-ordinator in July which meant that the P.C was unable to undertake certain activities due to lack of staffing capacity126

126 personal communication V. Hawkins

49 MSF Holland Afghanistan Evaluation Internal use only

8. Advocacy

This evaluation has observed MSF advocate on a broad range of issues in Afghanistan including; the need for food aid, protection of refugees and IDPs, conditions of repatriation for refugees, impartiality of humanitarian assistance from political goals (SFPCP), relocation of refugees to more appropriate sites in Pakistan, blurring of military and humanitarian mandates.

It appeared that in some cases MSF prioritised the need to be able to collect information for advocacy127 and the need to be strategically positioned (e.g. paediatric ward) over the need to respond to the health need of the most vulnerable (in health terms). Some of the advocacy issues and output were of more theoretical than practical use, e.g. the MSF position on the strategic framework. Advocacy and witnessing during the Taliban rule was reportedly on issues like mine awareness, training to include women, women’s access to Kabul hospitals, negotiation with the Taliban to retain national staff, responding to DFID policy of aid conditionally. More public advocacy actions were primarily directed at WFP and the international community while the approach taken with the authorities was much gentler. In some cases (e.g. the letter to the UN special envoy and others) the responsibility of the authorities is not adequately acknowledged128. This has been described as adopting ‘a bunker mentality’ towards the Taliban or viewed by some, as ‘taking a shot at the easy targets’ who do not have ultimate responsibility. And at the same time it is understandable that during the Taliban period that MSF were reluctant to risk jeopardising their relationship with the Taliban and therefore adopted a less strident tone on advocacy issues.

In spite of the fact that HAOs and a medical anthropologist had joined the programme before the period covered by the evaluation there appears to have been lack of clarity and focus with regard to priority areas for advocacy. It has been argued by some that with a clearer advocacy strategy with regard to women, greater inroads could have been made on issues of women's health129.

One issue raised by MSF was the blurring of military and humanitarian activities. MSF banned their staff going to ICRC parties (where military were often present). This tended to fragment the international community130. While it is was certainly a principled position, it may have had adverse effects on MSF staff, who were to a degree alienated and lost out on recreational activities. Also, it was not clear what the hoped for outcome would be. It was successful in raising the issue of ‘blurring military and humanitarian mandate’ for debate but the reasons behind the MSF stance were not clearly understood by some agencies interviewed131. With hindsight it may have been that there should have been more follow up with clearer reasons as to why it was felt necessary to take this approach. Advocacy actions on US Military food drops and refugee repatriation were positive.

The type of data used to advocate were variable in quality and credibility. For example, the MUAC data used to demonstrate either the need for food aid (targeted at WFP) or the need for revising the registration and food distribution system in Maslakh camp (MUAC survey) were probably inappropriate and lacked credibility. Most humanitarian actors would have paid far more attention to weight for height survey data. Furthermore, in identifying whom to target with an advocacy message in order to facilitate some form of intervention, it is critical to understand the room for manoeuvre or capacity which that agency has for response. It could be argued that targeting WFP with advocacy messages at a time when they were short of resources was of limited use and that efforts directed at the wider donor community and potential IP agencies (including those not operational in the areas in question) would have been more appropriate. Though this point was made by at least one MSF HOM132 the advocacy strategy often identified WFP as a more significant target. One of MSF’s advocacy positions was that WFP were not discharging their responsibility to ensure effective food distribution programmes and that this made them a legitimate target as it could be argued that more effective use of food aid resources would have increased availability of food for those who needed it . However it is noteworthy that WFP were never asked by MSF to reduce food going to the camps in the interest of directing it to more vulnerable areas.

127 repatriation of Afghan Refugees from Iran; May '02 proposal, EDR proposal 2001 128 initially the letter was to be copied widely however after protestation by UNHCR it was not 129 Personal Communications Oliver Matthieson 130 Personal communications IAM 131 WFP, IOM 132 Communication from Fouad Hikmat to project staff 28.5.01

50 MSF Holland Afghanistan Evaluation Internal use only

The data MSF collected from refugee interviews was useful in helping the international community understand the political circumstances and livelihood status of those fleeing to camps in Pakistan and border areas.

The process of advocacy is critical to likelihood of success. Success is more likely if advocacy is based on good data that results in consensus building between advocate and those being lobbied. Although, while it is difficult to judge from the outside, there were signs that advocacy around protection, registration and food distribution issues in Maslakh may have been more effective if handled with more sensitivity 133. There may also have been issues around the way in which MSF lobbied UNHR and the government of Pakistan (GoP) on refugee issues connected to the Chaman and Spin Boldak camps. For example, it is not clear whether MSF initially understood the full extent of the constraints on UNHCR and GoP with regard to locating appropriate sites for refugees after September 11th. This is not to say that the advocacy messages were not appropriate and well thought through but rather that the advocacy process could have been more considered. With better understanding a more consensual form of advocacy might then have taken place. In some situations it may be appropriate for MSF to ‘go public’ and try to ‘embarrass’ agencies into more appropriate action.

There have also been some concerns raised over the issue of HAOs who are employed specifically to help with the advocacy process. While invaluable in collecting appropriate information their very presence means that there is a risk of prioritising finding problems. Furthermore, the lack of experience/maturity of some HAOs has been raised as a concern in Afghanistan.

Relevant actors have not always been informed of public advocacy actions, e.g. the press release on Maslakh134. Consequently, for some agencies, first knowledge of the survey results was obtained from the Internet after a press release. This had negative effects on the collaboration between agencies on the ground. It has also been asserted that because of perceived lack of sensitivity in MSF’s advocacy strategy some agencies were less willing to share information with MSF135.

MSF's programmes appeared to be compromised as a result of conflict between the need to mount an effective programme based on assessed need versus the need to be in a location to collect information for advocacy/temoinage purposes. For example, the location of supplementary feeding programmes in open situations was arguably not justified in terms of responding to demonstrated need in those locations. Also, MSF appeared to prioritise an advocacy role connected to the Iranian and Pakistan returnee programme over assessed health needs. MSF has both medical and teimonage roles, but clarity is needed on whether health interventions are intended primarily to address the most important health needs or whether MSF should establish health programmes for a number of objectives including the need to be in a position to witness and to advocate. In short the primacy of the medical or the humanitarian role is not clear. The MTP does not help to clarify this. Consequently, priorities have varied according to individuals in charge (HOMs. OD). e.g. to meet the most urgent health needs, monitor human rights abuses, to be the watch dog of the UN, to respond to health needs which emerge from ‘crisis’.

8.1 Conclusions

• MSF's position in Afghanistan is critical given its independence and temoinage role particularly in the context of the compromised humanitarian space and the UN's proximity to the new government • serious consideration should be given to the role of HAOs proximity to projects given the likelihood that, under close observation, problems will be found • the advocacy process should demonstrate an understanding of constraints facing the agencies being lobbied • There should be a process of engagement with those who are targeted with advocacy messages and at the very least targeted agencies should be informed prior to public statements • credible indicators amongst the wider humanitarian community should be adopted to strengthen advocacy messages

133 personal communication IOM,HCR, WFP and MSFF. 134 However one source asserted that information was shared at a UNICEF co-ordination meeting. 135 IOM, WFP

51 MSF Holland Afghanistan Evaluation Internal use only

• an analysis of whether the agency being targeted with information would accept and (could) respond (buy-in) to the type of information being collected by MSF for advocacy purposes should be undertaken • advocacy objectives should be sufficiently specific and whether these objectives are being achieved during the course of the advocacy programme should be tested, e.g. the use of BFD information to trigger GFD programmes • the advocacy agenda should evolve out of medical programmes. Therefore those implementing programmes should set the agendas while HAOs should act as ‘resource people’ supporting the advocacy process

52 MSF Holland Afghanistan Evaluation Internal use only

9. MUAC versus Weight for height

For many years international consensus has been that MUAC measurements should only be used for screening purposes136 and that although MUAC may be a better indicator of outcome (in mortality terms) weight for height is the preferred indicator for assessing prevalence of acute malnutrition137.

Throughout the period evaluated in Afghanistan MSF believed that weight/height surveys were not culturally acceptable and that cold weather prohibited children from removing their clothes for weighing. As a result the only weight/height surveys conducted by MSF were in Shamshatoo refugee camp in Nov 2001 and Roghani (Chaman) refugee camp in January 2002.

MSF Draft guidelines138 state that MUAC can be used to: • identify individuals at risk of death • survey: rapidly detect malnutrition in a population determine the need for wgt/hgt random surveys • define admission criterion for TFC(MUAC<110mm) for children between 1and 5 years • screen: children pre-selection, determine which cases need additional assessment with wgt/hgt measurement

The target population for rapid assessment is quoted as 6 months to 5 years or 65cms to 110cms in height yet the guidelines acknowledge that MUAC is most reliable for children between 75cm and 110cm in height (aged 1-5 years). The guidelines also acknowledge that clear and universally agreed reference values for interpreting MUAC rapid assessment results do not yet exist.

MUAC was carried out on thousands of children during vaccination campaigns and during BFDs and to indicate prevalence of malnutrition in a survey in Maslakh camp.

It is known that MUAC and weight/height do not identify the same individuals as malnourished. This was confirmed on numerous occasions in Afghanistan. However, MSF have data and analysis from surveys which leads them to believe that prevalence of acute malnutrition can be accurately determined through the measurement of MUAC. Furthermore, MSF also maintains that results are comparable at specific cut-offs with the weight /height indicator139 (i.e. the prevalence of malnutrition as measured by MUAC <12.5cms will be roughly the same as if measured by <80% weight/height.)

There is some evidence that the relationship between weight for height and MUAC is population specific and may reflect different statures (which could be in part a reflection of endemic chronic malnutrition). It is not therefor appropriate to assume that the same cut-off point is equivalent for less than 80% weight for height for all populations. Afghanistan has very high rates of chronic malnutrition (>50%) and this may affect the specific cut-offs at which MUAC and weight for height are comparable. The inclusion of under 1year olds may also affect the relationship (prior to 2002 results from MUAC assessments including <1year olds estimated much higher rates than when <1year olds were excluded by age or height cut-offs.)

ACF conducted a series of surveys in 2000140 measuring MUAC and weight /height (see table). MUAC underestimated malnutrition prevalence. This is intuitive, as the height cut-off for MUAC measurement was <75cms, thereby excluding many children who might have been included on an age cut-off (as many would have been short for age). In addition, the MUAC cut-off used by ACF was <12cms rather than 12.5cms as is used by MSF.

136 ACC/SCN 1994 137 Briend et al 1989 138 MSF Draft guidelines March 2002 139 Personnel communication Saskia Van der Kam (data was not reviewed as part of this evaluation) 140 Compiled surveys Afghanistan 2000 ACF

53 MSF Holland Afghanistan Evaluation Internal use only

Table4: Rate of acute malnutrition rates, children 6-29 months, expressed in Z-scores, Wgt/hgt %of median and MUAC<12cms(children above75cmsof height) Afghanistan, 2000139 Survey / assessment141 Z-scores MUAC <12cms % of the median GMR (%) GMR (%) GMR (%) Kabul, February 2.8 0.4 1.6 Herat, July 7.3 0.6 5.0 Faizabad, Sept. 8.9 2.5 6.2 Kabul, October 8.0 2.7 5.1 Mazar, November 5.6 0.8 3.9 Jalalabad, Dec. 5.8 1 4.4

The survey conducted in April 2002 by UNICEF/CDC142 used an age cut-off of 1year old for MUAC measurements. This revealed a much higher prevalence when compared with the <80%of median weight/height cut-off (7fold increase).

In the MSF B survey data was collected on children between 65-110cms. MUAC, weight and height were measured on all children and prevalence according to both indicators varied.

In a survey conducted in Roghani (Chaman) camp by MSFH143 the prevalence of acute malnutrition as defined by MUAC was higher than that as defined by the weight/height % of median (by only 1.5 percentage points). MUAC measurements were recorded on all children in the sample (65-110cms).

Table5: Comparison of Weight/Height and MUAC from surveys in Maslakh141 and Chaman142 Champs Survey Z-Score MUAC<12.5 %of the median GAM (%) GAM (%) GAM(%)

Maslakh UNICEF/CDC 2.9 11.8 1.7

Faryab Province MSFB 9.8 12.9 8

Chaman MSF 7.1 7.1 5.6

Data reviewed by the evaluator collected in May and June 2002 in one of the AHDS clinics on all children between 1-5 years revealed that the two indicators identified similar proportions of children:

31% of the 130 children measured were identified as <80% weight/height compared with 35% identified as having a MUAC <12.5cms

In 2001 all MUAC screenings on thousands of children resulted in very high proportions >12.5cms 10- 30%. These MUACs were carried out on children of 6 months to 5 years (no height cut-offs were used). In 2002 MUAC was measured on children attending the BFDs however a height cut-off of 65cms was used144. Results were vastly different: 4.9% were found to have MUACs < 12.5 cms. A survey145 conducted in the province of Badghis in March 2001 using weight for height measurements showed similar rate of global acute malnutrition (5.5%). All children aged 0-5 years were included in the survey.

141All the surveys were done using the same methodology (30 clusters of 30 children), except for Kandahar, where the results are based on a sample of 13 clusters, including 403 children. 142 UNICEF/CDC Survey Maslakh April 2002 143 Nutrition and Health Survey Report, Roghani Afghan Refugee Camp Chaman, Pakistan January 6- 12,2002 MSF-H, Pakistan, Islamabad 144 Balamorgab BFD MUAC results, Dec 2001-Jan 2002, MSF-H 145 Nutrition and Health Survey, Badghis Province, Afghanistan, February – March 2002.

54 MSF Holland Afghanistan Evaluation Internal use only

9.1 Conclusions

• weight/height percentage of median did not estimate the same prevalence of GAM in surveys reviewed where both indicators were measured. The correlation between weight for height measurements and MUAC requires further analysis in this setting before MUAC can be used with confidence in place of weight/height to define prevalence of malnutrition. • height cut-offs appear to influence comparability • if height measurements must be used then some of the benefit of MUAC measurement as a quicker and less invasive tool are lost • if height cut-offs are used children most at risk are excluded (i.e. youngest) • agencies including MSF managed to undertake weight for height surveys on this population • given the current state of knowledge and international consensus, MUAC ‘screenings’ should not be used to estimate global acute malnutrition • MSF should treat with caution surveys and rapid assessments which estimate prevalence of acute malnutrition using MUAC cut-offs until a sound basis for its use is fully researched and disseminated • MSF may wish to research this issue and if necessary challenge and develop the international consensus

55 MSF Holland Afghanistan Evaluation Internal use only

10. Cross cutting Conclusions and Recommendations

10.1 Cross cutting Conclusions

In view of the extremely difficult working environment in Afghanistan over the period evaluated MSF’s achievements have been impressive. During the period in question MSF in Afghanistan responded to disease outbreaks swiftly and effectively, implemented thoughtful and comprehensive programmes and vaccinated thousands of children against measles. MSF undoubtedly saved lives. MSF were also flexible in approach and increased capacity in order to respond to a food crisis brought on by the drought. MSF used its position to speak out for humanitarian principals and against abuses of human rights. MSF volunteer’s level of commitment is beyond question while the commitment and capacity of MSF National staff was of the highest order.

10.1.1 Appropriateness of response

MSF’s primary analysis of the impact of the drought was in keeping with many other agencies, i.e. that there was a developing food crisis, which required emergency food and nutrition interventions. As such this became a priority focus of MSF's response to the drought. However the significant health crisis which had been ongoing for years in Afghanistan was being exacerbated by the drought. MSF's overall analysis emphasised food aid as the most appropriate response to the effects of the drought and sought to influence food aid agencies to provide GFD. MSF’s response was largely comprised of food and nutrition programming as well as the collection of data for advocacy (much of which was related to the need for food aid).

MSF could have capitalised more on their comparative advantage in the health sector by analysing and highlighting the health component of the crisis (where data is available it can be seen that high mortality was disease related and that acute malnutrition was low) and by focussing their interventions more fully on the health sector, i.e. investing more resources in further strengthening and expanding health programmes (EPR and PHC) and integrating nutrition components (where necessary) into these health programmes. Given MSFs current core competencies, this may have been a better option than focussing on a sector (food aid) where MSF were dependent upon the commitment and response capacity of other agencies. This does not negate the validity of advocacy actions to improve food aid. The massive measles vaccination campaigns and EPR as well as the EPREP are examples of where MSF did apply its emergency health capacity appropriately.

10.1.2 Assessment and analysis

Although MSF’s perception was that a GFD was the most appropriate response staff were conflicted about whether MSF should take an implementing role. Ultimately the decision was taken (based on many factors and previous experience) that MSF would not ‘do’ GFDs. Considering that there was a perception of dire need (the situation was described as a food crisis by the country programme) and the absence of other implementing partners, it appears that there was a strong internal position within MSF which led to this decision, which, however, is not in keeping with the MTP.146.

Most nutritional surveys undertaken during the period of the evaluation showed ‘unalarming’ levels of acute malnutrition. However, chronic child malnutrition was and is a serious public health problem in Afghanistan. Acute malnutrition was more evident in younger children. This was almost certainly related to diet quality and a heavy disease burden but requires further investigation. MSF did improve quality of diet through the SFPs and BFD although this was not the primary reason these programmes were undertaken (they were part of a broader objective to improve GFDs and to prevent a deterioration in nutritional status amongst affected populations).

Although MSF is not an agency with extensive experience and expertise in food security assessments, especially in the context of Afghanistan, it developed its capacity to analyse the food security in collaboration with Oxfam and as a result was able to undertake credible food security assessments. However as the food security situation did not appear to lead to the high rates of malnutrition

146 MTP p.12 ‘In case of dire need and on failure of others to provide, we may also be involved in the provision of potable water, sanitation, food and shelter'

56 MSF Holland Afghanistan Evaluation Internal use only anticipated this should have led to a more extensive investigation of the causes of high mortality and the determinants of malnutrition.

There was much need for credible information on how the drought was impacting the health and nutrition of the population. As a health agency with historical capacity in nutrition MSFs could have played a bigger part in collecting and interpreting such information (e.g. health and nutritional surveys). This would have provided programmatic direction in areas where MSF were operating.

MUAC screenings undertaken without appropriate sampling (and including under 1 year olds) were interpreted as global malnutrition rates (GMRs) and yielded spurious acute wasting prevalence estimates, which ultimately served to confuse rather than clarify the interpretation of the nutritional situation. All MUAC screening undertaken by MSF found “alarming rates of acute malnutrition”.

10.1.3 Programmes serving multiple purposes

MSFs addressed humanitarian and health needs in Afghanistan. However programmes were often expected to serve multiple purposes (to monitor human rights, to provide information for advocacy, to provide a strategic location, to respond to the health needs of the most vulnerable). These multiple purpose strategies sometimes effected technical programme effectiveness as a rigorous objective assessment of health and nutrition need prior to response was sometimes overlooked. Medical programmes were sometimes used to satisfy mainly proximity objectives and nutrition programmes were sometimes used to satisfy mainly advocacy objectives. MSF's role in disasters is depicted by the MTP as intervening at the interface between humanitarian and medical crises. The need to satisfy both humanitarian and health needs sometimes led to interventions being implemented which were not addressing ‘the most important’ health needs. Health staff found this difficult. For monitoring and evaluation purposes there was an emphasis on technical indicators however for intervention criteria there was an emphasis on witnessing, advocacy and monitoring humanitarian principles and human rights. Project objectives were generally stated in health or nutrition terms and monitoring and evaluation systems established using health or nutrition indicators without accounting for ‘other’ objectives.

10.1.4 Approach to implementation in an ongoing crisis

MSF did not have policies which address many of the issues that arise in connection with working in the type of longer-term health crisis that exists in Afghanistan. Such a context raises issues on project management, how to work with, through or independently of local partners and infrastructure, the degree of dependence on expatriate staff, coherence of health strategies and methods of withdrawing from programmes. Arguably, this absence of policy led to a modus operandi that was not always appropriate for the context.

The high dependency on expatriate staff with sometimes limited experience and short contracts, for project management led to lack of continuity of approach. The difficulties experienced by MSF in HRM were often related to inability to fill positions by expatriates. Yet MSF's experience has shown that management capacity can be accessed locally. The experience gained during the evacuation period demonstrated that in some locations National staff were capable of project management. Programmes which did rely on national staff members for management and supervision were coherent and appropriate (e.g. EWS).

10.1.5 Decision Making Process

Although the decision making process within the programme was difficult to review (it was seldom articulated in writing) it appears that actions and the decisions underlying those actions were very much personality driven (HOMs and OD positions). Decision making points and processes were also hard to identify as projects were usually written up in technical terms so that rationales were often omitted.

There were differing and contrasting interpretations within MSF of needs arising out of conflict as well as interpretations of what MSF’s role should be. This led to contrasting strategies during the period covered by the evaluation e.g., a focus on cities versus rural areas, a focus on advocacy versus technical

57 MSF Holland Afghanistan Evaluation Internal use only efficacy, and a focus on working through local partners or independently. Assessment to determine which needs were arising out of crisis and which were ‘normal’ was not undertaken allowing for ongoing difference in interpretation.

The changes in approach although related to the changes in context were also heavily influenced by changes in personnel. MSFs approach has been determined primarily by changing HOMs and ODs with differing interpretations of the needs and MSF's role.

Technical Departments acted as advisors sometimes only after strategies were defined and sometimes only after technical interventions had been established.

Field level technical staff did tend to defer to HQ technical departments for instructions on small project matters and were reluctant to take initiative and make changes even when instructions to do so came direct form technical departments147. Rather than utilising HQ technical personnel to oversee quality through question and answer type management it might be more appropriate to have this function decentralised to field / regional level with adequate skills /experience staff. HQ level technicians to be more involved in the design phase through involvement in the proposal formulation and writing process.

10.1.6 Health programmes

The type of health intervention supported or established sometimes necessitated a different approach by MSF as was experienced through the Western Districts PHC (programme intervention was in partnership with a local NGO allowing complementation of both agencies strengths). MSF’s capacity support for national NGOs was generally successful and appropriate although there was an increasing tendency for the role to shift more towards that of donor as the programmes evolved. Working in national structures or in partnership with national NGOs required a different approach to that of working in discrete structures established and managed by MSF. Efforts were nonetheless made to work with the MoPH and local partners – MOUs exist for various activities.

There was also a tension between MSF interventions aimed at improving quality of health care and the need to be involved in the short term. For example, MSF provided a level of resources (staff numbers and salaries) which would never normally be provided which in turn led to difficulties in withdrawing or extracting the organisation from the programme. The strategy also undermined other services by attracting staff from those services. Given the short term intense nature of MSFs commitment these interventions had ambitious objectives and created dependency, causing difficulties when it came time for withdrawing support.

10.1.7 Nutrition programmes

MSF’s programme experience over the period evaluated highlighted the difficulties of implementing a standard therapeutic feeding programme and supplementary feeding programmes (in open settings) in Afghanistan. MSF also perceived problems in relation to implementing standard anthropometric surveys, e.g. based on weight for height measurements, and therefore adopted MUAC as an indicator for nutritional surveys and screening. This had potentially serious negative implications for advocacy and also generated information that could not be so readily used for programming purposes. The experiences suggested a need for developing country specific emergency nutrition guidelines and policies and also for adhering to internationally accepted standards unless newer or unusual protocols were agreed and accepted by other key stakeholders working in Afghanistan. Difficulties in finding appropriate field level nutritionists undoubtedly effected the programme quality.

10.1.8 Programme closure

Lack of clear and realisable outcome related objectives sometimes resulted in poorly defined ending points (SFPs in open situations, hospitals, BFDs). In programmes that did not have a ‘natural’ end

147 For example advice to change antibiotic treatment in TFCs was not taken for months. (Personal communications Saskia Van der Kam and Christa Hook)

58 MSF Holland Afghanistan Evaluation Internal use only point, cut off points were arbitrarily based on time rather than objectives achieved. This is frustrating for those implementing the programmes and leads to poor results of evaluations, which test whether programmes have achieved stated objectives.

Project evaluation was not systematically included as part of programme design leading to missed opportunity for institutional learning within the country and in other MSF country programmes.

10.1.9 Advocacy

As an independent humanitarian organisation MSF embarked on a comprehensive advocacy role in Afghanistan. Advocacy for human rights, humanitarian assistance and protection of humanitarian principles was generally well informed and in many instances made a valuable contribution to the needs of those suffering. The positioning on the strategic framework was a useful exercise in raising awareness both in MSF and the wider humanitarian community. However, there were also examples of failed advocacy or where effectiveness was constrained by the process and method used to advocate. For example, MSF were largely not effective in mobilising GFDs in areas where they were implementing BFDs. There were also questions raised regarding the advocacy process, which, even thought it cannot be proved, may have limited the impact of their advocacy messages regarding Maslakh camp for IDPs. The advocacy strategy did not determine whether the agency being targeted with information would accept and could respond (buy-in) to the type of information being collected by MSF for advocacy purposes. MSF also did not appear to specify advocacy objectives and to test whether these objectives were being achieved during the course of the advocacy programme.

59 MSF Holland Afghanistan Evaluation Internal use only

10.2 Recommendations

10.2.1 MSF General i) MSF should clarify its role vis-à-vis the primacy of health, medical and humanitarian aims in a simple and practical manner which supports field managers enabling them identify appropriate MSF interventions. This should be reflected in the MTP, disseminated and monitored ensuring a common view among decision-makers ii) MSF Policy should acknowledge that approaches to chronic health emergencies may be of a longer- term nature and different to those in acute ‘emergencies’. In this regard the MTP should clarify MSFs role in chronic health emergencies as well as its position with regard to working with and within national health infrastructure. iii) In countries where a longer-term presence is likely and desirable MSF should: • investigate models of National staff development to reduce MSF HRM constraints e.g. investigate potential for project management positions to be filled by National staff v) MSF should strengthen its capacity to develop and maintain institutional memory through • the development and maintenance of appropriate information systems at HQ country and project level • wide dissemination of lessons learned • institutionalising formal debriefing vi) MSF should elaborate its policy on when it will engage in activities which are not core competencies i.e. in particular food, shelter, water and sanitation to give a clear picture of what constitutes ‘dire need’. vii) Recognising the potential operational and programmatic conflicts between implementing effective technical programmes and advocacy, it is important that MSF develops tools to determine ‘net benefit’ of programmes with ‘dual purposes’ in order to examine and determine the relative priority to be given to programme effectiveness and advocacy. Such analysis may show where programmes should be established primarily on the basis of trying to meet real and quantified medical needs148 vii) Management and quality assurance: i) The process of approving technical proposals should include approval from the relevant technical department. ii) Objectives should be explicitly stated in proposals even where these do not relate directly to health activities iii) the link between quality assurance and experienced and skilled staff should be recognised. MSF should explore other means to ensure availability of suitably experienced nutrition staff. If MSF want to maintain nutrition credibility they may have to review methods of attracting and retaining skilled and experienced personnel e.g. institutionalising some form of apprenticeship or mentoring scheme, review of allowances in comparison to other volunteer agencies. . iv) MSF should ensure that consideration is always given to technical coordination with other agencies in country to ensure optimal technical practice, and where appropriate, a standardised approach

148 A net benefit analysis would attempt to assess the reduction in technical effectiveness of programmes as a result of including broader aims (e.g. advocacy) and balance this against the potential positive humanitarian impact of including an advocacy component. Other agencies with strongly mandated advocacy roles, e.g. Oxfam, have been attempting to develop such tools to guide field level decisions regarding the relative priority to be given to technical programme design and effectiveness versus advocacy.

60 MSF Holland Afghanistan Evaluation Internal use only

10.2.3 MSF Nutrition Division i) MSF Nutrition section should clarify the role of MUAC measurement as an organisational tool for programme planning and advocacy. ii) Unless the current consensus changes (as evidenced by new research) MSF should treat MUAC assessments and surveys with the appropriate caution. iii) MSF could undertake research to validate a role for MUAC measurements in determining prevalence of malnutrition and actively disseminate findings and build consensus around the use of this tool if research findings support such a role. iv) MSF should clarify its policy and strategy on implementation of general food distributions based on an analysis of past experience and the internal debate on this issue that has taken place over the past decade. This clarified policy should set out options on what to do in circumstances where food insecurity is severe and undermining MSF health programmes. This policy might be represented as a decision-tree. This would lay out possible steps and actions sequentially e.g. • obtaining credible information on severity of food insecurity and affect on malnutrition, • lobbying for general ration intervention (who and how), • attempt to find implementing partners (local or INGO), This would also elaborate on the circumstances for considering whether MSF become an implementing partner or bring in own food e.g. • level of food insecurity or malnutrition, and/or • if health programmes are failing as a result of food insecurity and/ or • BFD being overwhelmed v) Capacity should be developed at country level to allow programme staff assess the cost of GFD implementation and the logistical, management and technical requirements of implementing programmes in different scenarios. vi) If BFD is being initiated as a holding or lobbying strategy it should not be initiated with an arbitrary cut off point but should continue until there is a change in the food security situation or followed up with a GFD. This implies a capacity to analyse and detect changes in the Food Security situation and also has implications on areas and populations to be targeted i.e. limited and known so that a commitment can be made. This may also entail verifying the level of commitment and capacity of organisations like WFP to provide the follow up GFD. vii) The position of an MSFH nutrition co-ordinator needs to be given priority over the international focal point position which needs revised terms of references.

10.2.4 MSF Health Division i) MSF should elaborate a policy for health interventions in longer-term chronic health emergencies. This policy should have a clear statement on:

• how MSF relates to the local health infrastructure • approaches in different scenarios (e.g. within government structures or through partners or independently) For different scenarios guidance on approach should be developed: • set appropriate objectives • appropriate level of commitment in terms of period of contract and resources • appropriate quality of care, and • when and how to withdraw (this should be determined and agreed at the start) ii) MSF needs to increase capacity at HQ to support field level epidemiological analysis and increased competence in integrating food security, health status and nutritional status analysis and be able to support those in the field who undertake such analysis. MSF’s expertise and health competence gives

61 MSF Holland Afghanistan Evaluation Internal use only them a comparative advantage in analysing the role of health factors in nutritional surveillance or survey findings149 iii) The Health Division should explore was of developing and supporting senior MSF National Staff medical personnel responsible for medical programme continuity.

10.2.3 MSF Context Analysis/Advocacy i) For each issue identified there should be a clear advocacy strategy highlighting advocacy activities and expected outcomes. These can be prepared in collaboration with the HAO. These outcomes should be tested throughout the period of advocacy ii) Attempts should be made where possible to quantify extent of problems which are being highlighted in the advocacy message, e.g. proportions of people experiencing forced repatriation. iii) MSF should utilise data, which are credible to those being lobbied. This may require prior agreement about what type of data the agency required iii) MSF should inform targets of advocacy if public criticisms are to be made about them. iv) Public denunciation should be treated very seriously with all efforts made to ensure that data are reliable and that everything else has been done and failed. The effect of public denunciation as a strategy may reduce if used frequently. Efforts should be made if appropriate to bring other sections (and agencies) on board to strengthen public statements. v) In the light of context analysis field programmes should be supported to identify key or thematic areas for MSF humanitarian advocacy

10.2.4 MSF Afghanistan Programme i) MSF should develop a longer term country health policy (3 year) and a strategy in collaboration with Amsterdam, clearly outlining MSF's main focus in the country and where and how MSF will work as well as how it will relate to health infrastructure. MSF's humanitarian as well as its health role should be explicit. The policy should include an emergency preparedness and response component which would identify for Afghanistan what type of emergency response capacity MSF will maintain. There should be a statement covering Food Crisis and MSF's role. ii) MSFH future country policies and strategies should have a clear statement on how interventions relate to the national health infrastructure – this should be widely disseminated. iii) Conditions for stopping projects should be determined and agreed prior to establishing MoUs. When the decision to withdraw from a project has been made an agreement on phasing out should be established with relevant partners. iv) MSF should strengthen local staff capacity with a view to appointing national medical and project co-ordinators, planners and programmers in the medium term. Expatriate staff would then be able to fit into a National senior and middle management structure. v) MSF should encourage exchange of positive experiences between projects e.g. well managed IDW to PW or the approach of the EWS in the SW could be expanded to other areas of operation to include health facilities not directly supported by MSF.

149 Nutritional surveys can be designed to help determine the relative significance of underlying and immediate causes of malnutrition, e.g. where health factors may be a primary factor in high levels of malnutrition. For example, BMI measurements of adults and a greater focus on prevalence of disease may help clarify the significance of health factors as opposed to food security factors in the aetiology of malnutrition.

62 MSF Holland Afghanistan Evaluation Internal use only

vi) Afghanistan may well face other food crises in years to come. Disaster preparedness is therefore critical if MSF is to continue working in that country. MSF should therefore maintain and develop its food security assessment capacity, compile a resource base of surveys and assessments in operational areas, develop country capacity for conducting nutritional surveys, co-ordinate with the nutritional and food security agencies involved in surveillance and monitor the food security situation in operational areas. vii) MSF also needs to develop country nutrition policies and guidelines, which address country specific issues and practices that have emerged through experience of recent interventions: viii) These guidelines/policies should address: • the incorporation of Nutrition as an integrated part of health care systems • infant feeding issues as a part of safe motherhood programmes • the ongoing analysis of food security context, utilising tools developed as a part of emergency preparedness • infant feeding as part of nutritional rehabilitation centres for severe malnutrition • the unsuitability of establishing SFPs in open situations where terrain is difficult and access limited highlighting the potential advantage of mobile SFPs • the cultural unacceptability of traditional TFCs in Afghanistan and the potential for adapting protocols to more home based care of the severely malnourished • the need to employ weight for height measures in nutritional surveys (rather than MUAC which may ‘appear’ more suitable for reasons of ease of measurement)

63