070228 Conceptual framework for analysis of TRIAMS data A pathway to provide context to data analysis: Case study

Karin Källander & Johan von Schreeb

Division of International Health (IHCAR), Department of Public Health Sciences Karolinska Institutet, Sweden

Nevio Zagaria World Health Organization, Health Action in Crisis, Geneva, Switzerland

WORLD HEALTH ORGANIZATION Table of Contents:

Executive summaryIntroduction...... 2 Introduction ...... 3 Aim...... 3 Objectives ...... 3 Methodology...... 3 Target group...... 3 Principles...... 3 Data quality...... 5 Results...... Impact assessment...... 7 General impact ...... 7 Infrastructure...... 11 Progress...... 15 Infrastructure...... 15 Basic social services...... 16 Vital needs ...... 19 Livelihoods ...... 28 Correlation analysis of progress in infrastructure and coverage basic social services - a future step ...... 29 Discussion...... 32 Annex 1 – the Karolinska Institutet Analytical Tool (KAT)...... 34 Objective for the KAT ...... 34 Is KAT useful?...... 34 Step by step manual...... 35

1 Executive summary

2 Introduction Reliable data and information is a cornerstone for reconstruction efforts in the aftermath of a humanitarian catastrophe. It is imperative to have monitoring data on i.e. numbers of houses needed per location, number of houses planned and constructed and the population size. When such data is available it is also crucial to ensure that it is being utilised in the planning process. These efforts are the underlying motives to why Karolinska Institutet (KI) is seconding WHO with technical support in implementing the Tsunami Recovery Impact Assessment and Monitoring System (TRIAMS). TRIAMS is based on selected indicators mainly collected as routine data by the authorities in the affected countries, with additional over samplings from Tsunami affected districts. During the 6 months that KI has been involved in TRIAMS we have noted that a lot of data is available but that efforts are needed to “harmonise” the data and also provide a simple tool to illustrate the analysis. In this document some of the challenges in regard to data collection, compilation and analysis are highlighted. Also a simple tool for visual analysis of data (Karolinska Analytical Tool, KAT) is introduced. We hope that our efforts can be useful for those involved in the difficult reconstruction work in the Tsunami affected countries.

Aim To provide a conceptual framework for the analysis and illustration of TRIAMS indicators of impact and progress using proportions and weights of affected populations, changes over time, and differences across administrative levels, using data from province in Indonesia as a case study.

Objectives • Introduce basic principles, assumptions and considerations with regards to analysis of a selection of TRIAMS indicators in the areas of Basic Social Services, Livelihoods, Vital Needs and Infrastructure. • Introduce methodologies for visual analysis of tsunami impact/destruction and recovery data at different administrative levels. • To apply these methods to illustrate impact and progress in Aceh Province, Indonesia and identify major trends and shortfalls in the recovery process.

Methodology

Target group Government officials involved in planning at district and provincial level

Principles In the aftermath of a disaster three main categories of data is urgently needed to plan relief activities. (A) Pre-disaster data on population size, administrative division, distribution and function of health facilities, schools etc (B) Disaster impact/destruction data. (C) Population based needs data of the affected population. However, data collection is too often considered of less urgency compared with direct relief efforts. Following the Tsunami, the initial lack of data was gradually changed to the opposite. Currently there are a number of databases at different agencies with vast amount of

3 data on various aspect of the recovery process. The challenge is to set up a systematic system that can make use of this data for in-depth analysis on sub-district level and monitor changes over time. In this conceptual framework we have used maps, tables and graphs to illustrate the effects of the tsunami and the progress of recovery. The graphing tool - Karolinska Analytical tool (KAT) - was developed for the purpose of providing a multilayer data analysis, based on both numbers and proportions (numbers/size of population). Numbers are important as they are directly linked to impact/destruction and output of services such as reconstruction of houses, healthcare facilities needed etc. Proportions are essential to contextualise data to population size and make the variables comparable across different groups/regions. For example, sub-district A with a population of 1 000 might have 300 dead while B with a population of 10 000 has 1 500 (). The numbers of dead are five times higher in sub-district B while the proportion of deaths (per population) is two times higher in sub-district A. The planning must take into account, not only the numbers of destroyed buildings but also the proportion of destruction as this is important in at least two different ways. First, if tsunami destruction of infrastructure was significant in combination with a high number of deaths, it could mean that the numbers of new schools, houses and health facilities needed are significantly lower than pre-tsunami (there are few persons left). While, if destruction was significant in combination with few deaths the needs for new construction is likely much higher. The proportion of villages destroyed and death rates in a sub- district must be part of the planning and monitoring. Migration of people in or out of districts is also very important to estimate as this also affects the needs.

Transparency is a guiding principle, clearly presenting assumptions, data quality and un-clarities

Figure 1. Two graphs that illustrate the difference between numbers and proportion in two districts, A and B.

70.0 85.0 % dead 1 600 300 1 500 No dead 1 400 100

1 200

1 000 % dead 800 50 % surviving 600

Number of deaths of Number 400

200

0 0 % of pre-tsunami population population pre-tsunami of % AB A: 1000 population B: 10 000 population District name 1 000 10 000 Pre-tsunami population

Another method that will be used to analyse impact and recovery are maps of districts and sub- districts. By disaggregating data by sub-districts, uncovered pockets can be identified and areas with slow progress can be highlighted (Figure 3).

Figure 2. Map of districts in Aceh Province, Indonesia.

4

Figure 3. Maps showing data aggregated by district level (A) and by sub-district level (B). The difference across sub-districts is apparent when disaggregating the data to smaller administrative units.

A B

Data quality TRIAMS data are collected by a large variety of ministries, agencies and, ultimately, people. In the process KI has received a vast amount of data to analyse. There are a number of problems involved in analysing such secondary data. For example methodologies on how data was collected are seldom described, sample size is unknown, time period covered is unknown, definitions of the

5 indicators are unclear etc. As a result, different data sets that report on the same indicators may show large variations in numbers. It is also impossible to know what a zero or a dash means in a data column of e.g. house reconstruction. Does it mean that zero needs/houses were built or does it mean that information was not available or that information was never collected? Such un- clarities should urgently be addressed to improve data quality. Due to this fact we have had to make certain assumptions to allow analysis. Some of these assumptions may be incorrect but without such assumption analysis would not be possible. In cases where assumptions have been made, sources of reference are provided.

As the uncertainty range for available data most likely is wide caution is needed when interpreting the results. Triangulation (similar type of data is obtained from different sources using different methods) is needed to validate some of the findings. For example, if data on house reconstruction in a sub-district is unclear or contradictive, remote sensing may be used to estimate number of houses constructed.

As the reliability of data is key more efforts are needed to improve the quality of data collection. Data should be accompanied with information on who colleted the information, when it was collected, what methodology was used and how was sampling done. If such information is available the quality of the analysis can improve.

6 Findings

Impact assessment

General impact The tsunami’s impact differs enormously from one country to another and, within each country, from one district or sub-district to another. Figure 4 shows the pre-tsunami population of each of four selected districts in Aceh province, Indonesia, together with the number of people who died or are missing as a result of the tsunami. This graphic presentation may help to visualize the weight of each district. The data is presented both in absolute numbers and as percentages. While absolute numbers are crucial to quantifying the devastating effects of the tsunami, they need to be complemented by rates, using the population figures as denominator, in order to capture the extent of the destruction in relation to the size of the affected area/population.

Figure 4. Dead or missing in numbers and as proportion of pre-tsunami population in four selected districts in Aceh province, Indonesia

35.0 21.3 8.6 5.4 1.7 1.2 1.0 0.5 % dead or missing 107342 16874 13 785 15394 1942 2652 5278 2316 Number dead or missing 100

Dead+missing

50 Alive Proportion of pre-tsunami population (%)

0 District name Aceh Besar Aceh Jaya Aceh Selatan Pidie Aceh Utara Pre-tsunami population 306 716 79 218 283 819 216 479 527 869 495 380

Aceh Barat Nagan Raya 160 755 111 656

7 0.5 0.1 0.1 0.1 0.1 0.0 0.0 0.0 % dead or missing 133 519 979 200 45 26 52 3 Number dead or missing 100

Dead+missing

50 Alive

Proportion of pre-tsunami (%) population 0 Sabang Bireuen Nias Aceh Singkil Aceh Barat Daya District name 28 657 350 611 739 118 156 079 147 018 118 259 Pre-tsunami population

Simeulue Aceh Timur 78 126 316 535

Interpretation: Aceh Barat and Aceh Jaya had similar death tolls, but the percentage of death in each district varies considerably, from 35% in Aceh Barat 5.4% in Banda Aceh. Such analysis should be taken into account in the planning of recovery efforts – in setting up the targets and timing of interventions, in estimating the overall resilience of the affected communities, in choosing among different temporary solutions, in influencing crucial elements of quality, and in determining the speed of the recovery process. Data quality: Relatively few deaths were recorded in Banda Aceh. WHO and the Karolinska Institute tried to validate this data through those involved in its collection. It transpired that the number of deaths and missing persons in Banda Aceh district only represented missing persons, while confirmed deaths were included in those of Aceh Besar district where the bodies were found This highlights the importance of ensuring the data reliability, despite the difficulty of collecting it in the immediate aftermath of a major natural disaster. Source file: NAD_IND_ALL_LOCAL.xls

Figure 5 illustrates the difference in pre-tsunami population compared to post-tsunami population. The difference in population observed likely result from a combination of deaths and migrations (both influx and outflux).

Figure 5. Pre-tsunami population compared to post-tsunami population in four selected districts of Aceh province

8 23.4 -18 558 37.3 11.5 10.1 6.4 4.4 3.8 3.3 2.4 2.4 % population change -24 940 -105 938 -53 510 -10 305 -20 326 -11 892 -10 175 -2 583 -6 717 Total population change 100

Death+migration

50 % of pre-tsunami population Proportion of pre-tsunami population (%) population pre-tsunami of Proportion 0 Banda AcehAceh Selatan Pidie Aceh Barat Nias Aceh TimurAceh Besar Aceh Barat Daya District name 276 960 Pre-tsunami population 283 819 216 479 527 869 160 755 462 158 316 535 306 716

Aceh Jaya Nias Selatan 79 218 118 259

0.0 % population change 0.2 12 087 Total population change 2.2 0.9 0.3 -60 0.0 0.0 0.0 0.0 0.0 -2 583 -1 445 -1 710 263 1 224 1 259 1 204 3 140 100

Death+migration

50 % of pre-tsunami population

0 Proportion of pre-tsunamipopulation (%) Aceh Barat Daya Aceh Utara Sabang Bireuen Aceh Singkil Aceh Tamiang District name 118 259 495 380 28 657 350 611 147 018 232 174 Pre-tsunami population Lhokseumawe Simeulue 156 079 Nagan Raya 78 126 136 382 111656

Interpretation: The significant change of population in Banda Aceh can be explained not only by migration but also by the fact that Banda Aceh deaths were not recorded for this district (see interpretation in figure 3). Source file: populationAceh.xls

9 Figure 6. Number of dead+missing and as proportion of pre-tsunami population in six villages (desa) of Kuta Raja sub-district (kecamatan) of Banda Aceh district

69.6 67.5 65.5 65.3 64.4 59.0 % of population dead/missing 835 3 387 3 515 1 787 2 147 1 993 No of dead/missing 100

Dead/missing Alive 50 % of pre-tsunami population pre-tsunami % of

0 Lampaseh Kota Merduati Keudah Peulanggahan Gampong Jawa Desa name 5 017 5 365 2 738 3 336 3380 Pre-tsunami population Gampong Pande 1199

Interpretation: As the area of analysis is narrowed down to village level (desa), the effect of the tsunami becomes much more homogeneous, as in this urban sub-district where 35% of the population survived. This will particularly be the case for the worst-hit areas. A completely different distribution of deaths and missing persons emerges compared with the analysis on district level (see figure 3). Source file: Kuta Raja kecamatan office report

Figure 7. Number of persons living in severely- or non-affected villages as proportion of the population in sub-districts of Aceh Jaya

10 83 67 58 52 44 35 % of population in severely 7 095 3 775 9 646 2 491 7 186 3 846 affected villages 100 Number of people in severely affected villages

People in severely affected villages 50 People in non-affected villages

0 Sub-district name Proportion of pre-tsunami population (%) Krueng Sabee Jaya PangaTeunom Sampoiniet Setia Bakti Post-tsunami population 8599 5594 16673 4816 16285 10891

Interpretation: There is a significant variation across the sub-districts of Aceh Jaya, ranging from 35% to 83% of the population living in “severely affected” villages. The destruction was particularly severe in Jaya and Krueng Sabee sub-districts where 67 and 83% of the population live in severely affected villages. The resilience of the affected communities may be very different. The strategy for the implementation of the different recovery interventions needs to be adapted accordingly. Data quality: Data from other districts categorize villages as “moderately” and “slightly” affected. No explanation is available for the different terminology adopted across the districts. Source file: Kecamatan_Podes2005_NAD_HEALTH.xls

Infrastructure There are significant variations in the number of recorded deaths and missing persons from one area to the next, but the overall level of destruction and economic loss is not necessarily proportional to the number of deaths. With respect to the number of people displaced or houses destroyed Indonesia and Sri Lanka face similar recovery challenges. However, where the impact may have been similar, often it is the capacity to recover that varies greatly between neighbouring districts or sub-districts.

Figure 8. Number of persons with houses destroyed as proportion of the population in all affected districts, Aceh province

11 63 12 59 48 43 40 31 12 10 8 % of population with damaged house 49718 436285 37506 69709 112063 5658 14369 13273 33 776 41 889 People with damaged house 100

Population with damaged/destroyed house 50 Population without damaged/destroyed house

0 Aceh Jaya Nias and Nias SelatanSimeulue Banda Aceh Aceh Besar Aceh Utara District name Proportion of pre-tsunami population (%) population Proportion of pre-tsunami Nagan Raya Bireuen 495 380 Pre-tsunami population 79218 739118 78126 283819 306716 111656 350 611 Aceh Barat Aceh Barat Daya 160755 118259 7 7 6 5 4 3 1 1 0 % of population with damaged house 9854 10358 7014 26180 5069 5658 4113 1446 556 People wtih damaged house 100

Population with damaged/destroyed house 50 Population without damaged/destroyed house

0 Proportion of pre-tsunami population (%) population pre-tsunami of Proportion District name Aceh SingkilNagan Raya Pidie Aceh Barat Daya Aceh Timur Aceh Tenggara Pre-tsunami population 147018 111656 527869 118259 316535 232174

Lhokseumawe Aceh Selatan Langsa 156079 216479 136382

Interpretation: The destruction was particularly severe in Aceh Jaya and Nias, with more than 60% of the population loosing their house. Data quality: Population affected = Number of households damaged/destroyed1 multiplied by average household size2 Source file: 1IOM Mar & Apr 2005 (housing data.xls); 2 VillagePodes2005

Figure 9. Number of damaged or destroyed Camat offices by district in Aceh Province

12 Number of Camat offices pre-tsunami 8 2 6 3 5 9 6 22 17 30 3 11 Number of Camat offices 825 2 3 535 7 7 2 0damaged/destroyed 100

Total damaged or destroyed

50 Not damaged

No info Camat offices post-tsunami (%) post-tsunami offices Camat 0 Name of district Post-tsunami population Pidie Pidie Bireuen Sabang Simeulue Simeulue Aceh Jaya Aceh Barat Aceh Besar Banda Aceh Nagan Raya Nagan Raya

78 126 Lhokseumawe Aceh Barat Daya Aceh Barat Daya

28 657 79 218 156 079 111 656 283 819 118 259 306 716 350 611 527 869 160 755 136 382

Figure 10. Rating of Camat office infrastructure and involvement in recovery, by district in Aceh Province

Good

Basic

Poor NIAS PIDIE SABANG BIREUEN ACEH JAYA ACEH BARAT ACEH BESAR ACEH UTARA NAGAN RAYA NAGAN BANDA ACEH ACEH TIMUR NIAS SELATAN KOTA LHOKSEUMAWE KOTA

Camat office infrastructure rating Camat office involvement in recovery rating

13 Definitions:

A) Camat office infrastructure: 1 Number of offices destroyed and still waiting for new construction and/or very limited or no staff in core positions (camat, sekcam etc.) 2 Basic office facilities not fully equipped, in need of repairs and/or not fully staffed in core positions (camat, sekcam etc.) 3 Well equipped and fully staffed office

B) Camat monthly meetings on recovery: 1 Rarely hold monthly meetings with recovery organisations and only have minor or no involvement in managing recovery 2 Often hold monthly meetings with recovery organisations and have some involvement in managing recovery 3 Always hold monthly meetings and often work bilaterally with organisations and communities to speed up recovery

RANGE Min Max Poor 0.5 1.5 Basic 1.5 2.5 Good 2.5 3

Interpretation: It is very difficult to draw any conclusions about current Camat functionality as there is a lack of baseline information. For example, in a district like Aceh Timur, where Camat office involvement in recovery currently is poor, we cannot tell if this is a result of tsunami destruction or any other reasons, such as the conflict. We also cannot tell whether the status has improved as a result of recovery efforts. For indicators such as this one, where no baseline is available, the progress will have to be monitored in relation to the latest available data - in this case the above rating from 2006.

Specific comments: Camat offices generally have a high level of infrastructure in place ranging from temporary offices to fully equipped permanent offices. Despite relatively recent establishment of BAFO, Camats generally are involved considerably in recovery process, aided by a number of organisations such as AIPRD. Jaya office: Basic infrastructure and medium involvement in recovery. The upcoming departure of UNDP UNVs may be an issue. Meulaboh office: involvement in recovery here is higher than the Camat office infrastructure, reflecting the long term support that the Meulaboh office have provided to the local government in their area. However there is a need to upgrade the Camat office facilities in order to ensure sustainability. Lhokseumawe office: Camat office infrastructure here is at a surprisingly high level, despite tsunami and conflict. However local government involvement in recovery activities is low. Nias office: Figures are lowest of all areas. This is partly because there have been 18 new sub-districts created recently and the Camat offices are still setting up. However, involvement in recovery of preexisting Camats is poor, reflective of the lack of engagement on the part of the BRR field office with local government. The poor performance needs to be raised both at the Nias level but also at the central Banda Aceh level.

14

Progress So far, the proposed conceptual framework has been used to illustrate the impact of the tsunami at the district and sub-district levels, correlating the different variables to the size of the denominator (e.g. number of deaths in relation to the size of the pre-tsunami population), presented both as absolute numbers and as proportions. The same model can be used to illustrate the progress of the recovery process using selected indicators at any one time or over a certain time period.

Infrastructure In this area of recovery, problems were exacerbated by logistic issues and the inability to deal with increased infrastructure demands. Countries found themselves with limited human resources skilled in construction to be able to respond properly to immediate and long-term needs. These challenges have been addressed in different ways as they vary in nature from one country to another. The most evident implication of logistic problems is cost. This is particularly true for areas along Aceh’s western and southern coasts and has contributed to a significant budget shortfall.

In Indonesia, the World Food Programme (WFP) and the Aceh and Nias Rehabilitation and Reconstruction Agency (BRR) collaborated to expedite much-needed supplies to Nias Island. The WFP Shipping Service, supported by the Multi-Donor Trust Fund, enabled implementing agencies to bring supplies to the island, thereby reducing the island’s isolation. The WFP initially provided this service free of charge but is now moving to a cost-recovery model. Opportunities to “build back better” were shared. Indonesia is using reconstruction activities to strengthen local governance, implement anti-corruption strategies and install transparency tools such as e- procurement. Strengthening community leadership during the recovery process has also been a key factor in the sustainability and success of reconstruction efforts.

Figure 11. Number of rehabilitated or constructed health units (puskesmas) by 10,000 population and district 88 84 50 50 32 25 25 22 20 14 3 0 % of heatlh units damaged/destroyed 71145371221210 0 Number of puskesmas damaged/destroyed 100 25

20 Puskesmas damaged/destroyed

15 Puskesmas not 50 damaged/destroyed (%) 10 Puskesmas built per 5 10,000 population

0 0 population 10,000 per built Puskesmas Proportion of pre-tsunamipuskesmas Aceh Jaya NiasAceh Barat Aceh Besar Nagan Raya Bireuen Pidie Aceh Barat Daya District name 60 660 712 075 150 450 296 541 304 643 148 277 493 670 115 676 Population post-tsunami

Banda Aceh Sabang Simeulue Lhokseumawe 191 539 28 597 78 389 160 549

15

Interpretation: Health facility reconstruction is here measured by new or rehabilitated units (Puskesmas) per 10,000 population in order to be comparable across the districts. Aceh Jaya has by far the largest number of new and rehabilitated health facilities per population, with 23 rebuilt/reconstructed health units per 10,000 population. This is good progress considering that the damage also was very severe in this district (88% of puskesmas were damaged or destroyed). On the contrary, in Nias, where 84% of the puskesmas were damaged/destroyed only 2 have been reconstructed or rehabilitated per 10,000 people. Aceh Barat and Banda Aceh, both with 50% of puskesmas damaged or destroyed, have had 6 and 3 puskesmas respectively reconstructed per 10,000 people. Simeulue, which only had 2 health units damaged in the tsunami, has a very high number of new or rehabilitated health units per population compared to the damage, with 10 reconstructed health facilities per 10,000 people. Source file: Dec 05 hc facilities damaged.xls (BRR/UNIMS data 2005)

Basic social services Damage to health and education facilities caused by the tsunami impaired the adequate provision of these services in its aftermath. Moreover, the loss of human life included large numbers of health personnel, leading to a shortage that further hampered health-care delivery. Coverage of measles immunization in children is a good indicator to illustrate the health system functionality (Figure 12).

100

80

60

2004 2005 2006

% of children 40

20

0

t n a h g a ¤ ¤ ¤ a ur a e sa r g n r y w g n a ay ue Pidie eue a n b J ngah l Bar Tim Da m a gga h h Bir h u n Sa e Te e rat e L ta Te Ac h Gayo Lues Ace Aceh Utara Ac Ba ks h e Simeu Aceh Singkil Nagan Raya o Ko e eh Tamia Aceh Besa Aceh Selatan h Bener Meria Ac Banda Aceh L Ac Ac Aceh Kota

Figure 12. Proportion of children 9-12 months immunized against measles by year and by district, Aceh Province

16 * Data for 2006 only available up to April # Data for 2006 only available up to July ¤ No data available for 2004

Interpretation: The immunization rates are reported cumulatively over the year for children in the target group 9-12 months. The data that is available for 2006 only goes up till August or September (April and July for some districts), thereby only representing the first 8-9 months of the year. The rates are therefore likely to increase in all districts by the end of the year. Two districts (Bireuen and Aceh Besar) have by August/September reached the immunization rates of previous years. Several others (Aceh Barat, Lhokseumawe, Sabang, Aceh Barat Daya and Simeulue) are within 10 percentage units from last years coverage rates already in August/September. Of the tsunami affected districts, three had lower immunization rates after the tsunami than before (Aceh Barat, Bireuen and Lhokseumawe), while two had almost equal rates (Aceh Jaya and Nagan Raya) and four had better coverage after the tsunami (Sabang, Aceh Barat Daya, Pidie and Kota Langsa). Data quality: August or September data is not available for Aceh Selatan, Aceh Singkil, Aceh Tenggara and Gayo Lues. As the rates are reported cumulatively it is not possible to compare these districts with others where data is more updated. There is no data provided for Nias. There is no baseline data (2004) for three districts (Banda Aceh, Aceh Besar and Simeulue). Source file: Social serveices.xls. Source agency : PHO

2004 2005

Figure 13. Percentage of children 12-23 months immunized against measles in 2004 and 2005 respectively.

17 Table 1. Number of hospital and puskesmas consultations per person per year in 2004 and 2005 District Hospital Hospital Chang Puskesmas Puskesmas Chang consultation consultation e in % consultation consultation e in % s per person s per person s per person s per person per year per year per year per year 2004 2005 2004 2005 Aceh 627 -43 0.014 0.103 0.644 0.369 Tamiang Banda Aceh 0.297 1.144 285 0.013 Simelue 0.152 0.459 203 0.812 Aceh Barat 0.068 0.199 192 0.469 Pidie 0.083 0.134 62 0.178 Aceh Utara 0.129 0.195 50 2.453 0.761 -69 Aceh Tengah 0.121 0.135 12 0.654 0.303 -54 Sabang 0.804 0.865 7.6 2.924 1.119 -62 Bireun 0.170 0.151 -11 0.985 0.720 -27 Aceh Selatan 0.094 0.013 -86 0.002 Aceh Timur 0.177 1.013 Aceh 0.057 0.204 Tenggara Aceh Besar 0.058 1.317 0.613 -53 Aceh Jaya 0.218 Aceh Singkil 0.595 0.303 -49 Lhokseumaw -29 0.000 1.140 0.805 e Nagan Raya 0.098 Aceh Barat 0.107 Daya Bener Meriah 0.362 0.361 Gayo Lues 0.176 Langsa 0.411 0.797 94 Nias* 0.412 0.746 81

Negative No data Positive change change * Includes both Nias and Nias Selatan districts

18

2004 2005

Figure 14. Number of hospital and puskesmas consultations per person per year in 2004 and 2005

Interpretation: While most districts have observed a positive increase in number of hospital consultations per person per year, the majority of districts report a negative change in puskesmas consultations. It is possible that this results from a large number of puskesmas being damaged by the tsunami, forcing patients to instead seek health care from hospitals. Although it would be interesting to explore the relationship between puskesmas consultation data and the proportion of puskesmas damaged in the same districts, this is not possible to do since baseline data only exist for 4 of the 11 districts which had puskesmas damaged or destroyed by the tsunami. Data quality: Given the great lack of data, especially from puskesmas in 2004, it is very difficult to draw any conclusions with regards to changes in outpatient consultations in hospitals and puskesmas following the tsunami. Source file: data TRIAMS 2004 2005 2006.xls (Collected by WHO in Nov 2006)

Vital needs Vital needs, particularly for water, food and sanitation, require immediate responses, which also have to be maintained into the early part of the recovery phase. Making the shift from temporary emergency solutions to permanent solutions, such as housing, is the key challenge. There are few indicators to show whether this process is happening homogeneously across different districts or sub-districts. The provision of housing (both temporary and permanent) is one important indicator that can be monitored both according to needs and over time. However, data is far from complete (many double-entries and a lot of information missing) and requires considerable effort and investment for improvement.

19

Figure 15. People in need of house in numbers and as proportions of the total population in the all affected districts in Aceh Province 2005

64 47 34 32 26 26 0 9 8 8 % of population in need of house 38805 37109 51622 56250 78090 5666 14725 13465 12056 26963 Number of people in need of house 100

Population in need of house

50 Population not in need of house

0 Proportion of post-tsunami population (%) population post-tsunami of Proportion Aceh Jaya Aceh Barat Banda Aceh Aceh Besar Nias selatan* Nagan Raya Lhokseumawe Bireuen District name 60 660 150 450 177 881 296 541 270 243 123 743 154 634 351 835 Post-tsunami population

Aceh Singkil Simeulue 148 277 78 389

7 6 5 0 5 3 2 % of population in need of house 35084 7361 8825 22739 929 5666 55 Number of population in need of house 100

Population in need of house

50 Population not in need of house Proportion of post-tsunami population (%) population post-tsunami of Proportion

0 Pidie Aceh Barat DayaAceh Selatan Aceh Utara Aceh Timur Langsa District name 474359 115676 191539 493670 304643 137586 Post-tsunami population

Sabang 28597

*Average household composition assumed to be 5,2 people (extrapolated from JAMA. 2006;295:1240-1244)

20

Figure 16. Number of houses needed by districts in Aceh Province 2005 and 2006

Figure 17. Number of houses needed by sub-district in Aceh Province 2005 and 2006

21 Interpretation: By multiplying the total number of houses needed in a district with the average household composition, we obtain an indicator for “population needing a new or rehabilitated house”. This indicator shows that even if the total number of houses needed is highest in Aceh Besar and Banda Aceh, the biggest need by population is in Aceh Jaya district, where 77% of the population is in need of a new or rehabilitated house. Also Simeulue and Aceh Barat have a higher number of people needing a new house than in Aceh Besar and Banda Aceh. It would therefore be recommended that housing reconstruction is not only assessed in relation to the total number of houses needed in a district, but also in relation to the proportion of the population who are in need of a new house. Source file: Population in need of house: house needs (BRR/Garansi Survey “dbSurvey20051121.mdb”) multiplied by average household size (VillagesPodes 2005.xls) with the exception of Nias Selatan where no household composition data could be found and instead we extrapolated the number 5,2 which was reported in a survey published in JAMA. 2006;295:1240- 1244.

Figure 18. Housing reconstruction in relation to housing needs and population affected by housing damage, by district in Aceh Province

150

12 1236 13500 2037 3066 84672737 1664 Number of houses needed 100

Unmet need Committed 50 Under construction Built Proportion of(%) houses needed

0 Aceh Timur Nias selatan* Simeulue Lhokseumawe Name of district Aceh Selatan Number of people in need of house 37109 12056 5666 70200 8825 Langsa 55 Aceh Singkil Aceh Barat Daya 13465 7361

22 231

150

11 207

6158 3866 13294 8585 15025 5319 19446 Number of houses needed 100

Unmet need Committed 50 Under construction Built Proportion of houses needed (%) needed houses of Proportion

0 Name of district Bireuen Nagan RayaAceh Barat Pidie Banda Aceh Aceh Utara Aceh Besar Aceh Jaya Number of people in need of house 38 805 26 963 14 725 51 622 35 084 56 250 22 739 78 090 Sabang 929 *Data only available for Nias Selatan

Interpretation: Reconstruction of housing is shown as a percentage of houses completed, those under construction, those already committed with available resources and those where the need remains to be covered. The area of each sub-district is based on the weight of affected population in need of a house. Data quality: It was discovered during the analysis that the needs data from the Garansi survey has entered more than one number for the “houses needed” in several villages. We therefore had to make the assumption that in villages with more than one “house need” reported, the numbers should be added up together. It could, however, be so that these numbers derived from different sources (i.e. different people’s estimation of house need in a village) and therefore should be averaged to get a final “need number” for the village. Until this can be confirmed, we decided to use the added sum instead of the average. Source file: Population affected: as in Figure 15 above; House needs: BRR/Garansi Survey “dbSurvey20051121.mdb”, Progress: “BRR April Data Pack Survey – published.xls”)

Figure 19. Housing reconstruction in relation to housing needs and population affected by housing damage, by tsunami affected sub-districts in Bireuen district

23 150 578

28 260 495 557 289 1 889 432 1 541 89 Houses needed 100

Unmet need Committed 50 Under construction Built Proportion of houses needed (%) needed houses of Proportion

0 Name of sub-district JANGKA PEUDADA SIMPANG MAMPLAM SAMALANGA GANDA PURA Number of people needing house 133 2 463 7 786 6 747 392 PANDRAH PEULIMBANG KUALA BIREUEN JEUMPA JEUNIEB 1 029 2 014 1 358 1 817 2 477

Interpretation: Reconstruction of housing is shown as a percentage of houses completed, those under construction, and those already committed with available resources and those where the need remains to be covered. The area of each sub-district is based on the weight of the affected population in need of a house. It is worrisome to see that the sub-district Pandrah has reported no housing reconstruction activities. No houses have been built, none are under construction and there are no commitments to build here, even though there are approximately 1029 people who lack a house. Data quality: See Figure 18. In addition, the graph shows the sub-district divisions and needs that were reported in 2005, after a number of new sub-district divisions had been added to Bireuen district. However, pre-tsunami population data comes from Podes 2003 data. Therefore, the pre- tsunami population in some of the sub-districts that received new administrative boundaries may not be completely accurate. Source: Housing needs: BRR/Garansi Survey “dbSurvey20051121.mdb”, Progress: “BRR April Data Pack Survey – published.xls”)

24 Figure 20. Houses reconstructed in May 2006 and November 2006 in relation to housing needs and population affected by housing damage, by district in Aceh Province

150

12 13500 3066 12362037 84672737 13294 100 Number of houses needed

Unmet need Built October 2006 50 Built May 2006 Proportion ofhouses needed (%)

0 Nias selatan Aceh Singkil* Simeulue Lhokseumawe Aceh Barat Name of district Number of people in need of house 70200 13465 37109 12056 51 622 Langsa 55 Aceh Tim ur Aceh Selatan 5666 8825

150 (%) d e d

11 207 3866 8585 15025 1664 53196158 19446 231 Number of houses needed 100 ouses nee ouses h f

on o on Unmet need ti 50 Built October 2006 Built May 2006 ropor P

0 Aceh Jaya Nagan Raya Pidie Banda Aceh Aceh Utara BireuenAceh Besar Sabang Name of district 26 963 929 Number of people in need of house 22 739 78 090 38 805 14 725 35 084 56 250 Aceh Barat Daya 7361 * District that reported a total number of houses built that was lower in October than in April

25

Figure 21. Houses reconstructed in May 2006 and November 2006 in relation to housing needs by district in Aceh Province

Figure 22. Houses reconstructed in May 2006 and November 2006 in relation to housing needs by sub-district in Aceh Province 26 Interpretation: Reconstruction of housing is shown as a percentage of houses completed in May and in October and those where the need remains to be covered. The area of each district is based on the weight of the pre-tsunami population. It is positive to see that by October, the housing construction has started in most of the tsunami affected districts. It is positive to note that housing reconstruction has taken off in almost all districts which had houses destroyed by the tsunami. However, progress is most intense in the districts which already had good progress in housing construction in April. Only in three districts, Nias Selatan and Aceh Singkil and Aceh Timur, has not had any progress since April. Data quality: The October data should be interpreted with caution since it was analysed before being thoroughly checked and cleaned by BRR who collected the data. Even if the exact numbers may be faulty, the general trends can still be used to analyse the situation. One district, Aceh Sinkil, reported having built 8 houses less in October than they reported having built in April. Source: Population affected: as in Figure 15 above; House needs: BRR/Garansi Survey “dbSurvey20051121.mdb”, Progress: “BRR April Data Pack Survey – published.xls” and “Recon October.xls”.

Figure 23. Houses reconstructed in April 2006 and October 2006 in relation to housing needs and population affected by housing damage, by tsunami affected sub-districts in Bireuen district 150

28 260 557 289 1 889 495 89 578 1 541 432 Houses needed 100

Unmet need 50 Built October Built May Proportionof houses needed (%)

0 Name of sub-district JANGKA KUALA BIREUEN SIMPANG MAMPLAM PEULIMBANG JEUNIEB SAMALANGA 1 358 Number of people needing house 133 7 786 2 014 2 477 6 747 PANDRAH *PEUDADA *GANDA PURA JEUMPA 1 029 2 463 392 1 817

* Districts that reported a total number of houses built that was lower in October than in April

Interpretation: Reconstruction of housing is shown as a percentage of houses completed in April and October 2006 and those where the need remains to be covered. The area of each sub-district is based on the weight of the pre-tsunami population. Disappointingly, there are still no housing reconstruction activities reported in Pandrah sub-district. Data quality: See Figure 20. Two sub-districts, Peudada and Ganda Pura, reported having built 10 and 39 houses less, respectively, in October than they reported having built in April. Source: Housing needs: BRR/Garansi Survey “dbSurvey20051121.mdb”, Progress: “BRR April Data Pack Survey – published.xls”)

27 Livelihoods Income-generating capacities were severely hit in almost every area affected by the tsunami. Not everyone affected by the tsunami has resumed their previous occupations; some have taken up new occupations whenever the support and/or the opportunity has been provided. In certain sectors, numerous professionals perished in the disaster (e.g. fishermen or local government workers). One indicator that illustrates progress in the livelihood sector is the number of fishing vessels that has been replaced or repaired (Figure 24).

Figure 24. Number of fishing boats damaged and repaired/reconstructed by district in Aceh Province 150

% of fishing boats reparied/rebuilt 1 5 5 616 19 25 27 30 35 Number of fishing boats 20 20 42 30 222 145 242 150 278 179 reparied/rebuilt 100

Unmet need 50

Fishing boats replaced/repaired

Proportion of fishing boats damaged (%) damaged boats fishing of Proportion 0 District name Aceh Tamiang Simeulue Bireuen Aceh Utara Nias 917 Fishing boats damaged 1577 846 1393 952

Nagan Raya Aceh Selatan Langsa Lhokseumawe Aceh Timur 783 561 512 409 524 166 240

150

65 208 70 % of fishing boats replaced/reparied 37 45 54 61 63 64 76 Number of fishing boats 247 605 246 216 830 424 100 replaced/reparied

Unmet need 50 Fishing boats replaced/repaired Proportion of fishing boats damaged (%) damaged boats Proportion of fishing

0 Aceh Singkil Aceh BesarAceh Jaya Sabang Pidie Aceh Barat Nias Selatan District name 659 1330 455 356 1321 662 145 Fishing boats damaged

Banda Aceh Aceh Barat Daya 318 108

28

Figure 25. Number of fishing boats damaged and repaired/reconstructed by district in Aceh Province

Interpretation: Overall, 32% of the fishing boats reported damaged or destroyed have been replaced or repaired. However, the number varies significantly by district. In 12 tsunami affected districts, less than 50% of the fishing boats that were damaged have been replaced or repaired. On the other hand, in Nias Selatan, more boats have been built than the number that was reported damaged/destroyed (166%). Data quality: It is very questionable whether the district Aceh Tamiang have in fact had 1577 fishing boats damaged, since this is one of the districts that was not much affected by the tsunami. This number could either be a misreporting or an error in the data entry. It is also questionable whether the number of fishing boats damaged actually can be used to define the “need” of new boats since many of the destroyed boats’ owners could have died in the disaster. But without any proper indicator for number of boats needed, this variable is the only one that crudely can define the target that should be achieved. Source: Damage data from FAO In The 1st Commemoration Report, Sep 2005. Progress data from BRR RANDatabase (Report up to October 2006)

Correlation analysis of progress in infrastructure and coverage basic social services - a future step

Figure 26. Proportion of primary school damaged/destroyed and primary school enrolment, by district for Aceh Province

29 100 100 100 99 99 99 97 79 77 72 59 55 % of SD schools damaged/destroyed 100 100

SD schools 95 damaged/destroyed SD schools with no 50 damage (%) SD enrolment rate girls 90 SD enrolment rate boys Schoolenrolment 2005 (%)

Proportion of pre-tsunami SD schools SD schools pre-tsunami of Proportion 0 85 Bener Meuriah Nias Simeulue Aceh Tenggara Aceh Barat Pidie Aceh Jaya District name 61 939 14 113 65 269 10 933 27 089 19 035 7 793 Population 7-12 years

Aceh Singkil Aceh Tengah Nagan Raya Banda Aceh Bireuen 25 034 22 045 17 094 15 696 45 975

51 47 44 38 26 23 12 9 3 1 0 % SD schools damaged or destroyed 100 100 SD schools damaged/destroyed

95 SD schools with no damage

50 SD enrolment rate girls

90 SD enrolment rate boys School enrolment 2005 (%) 2005 enrolment School

Proportion of pre-tsunami SD schools (%) SD schools pre-tsunami of Proportion 0 85 Aceh Utara Aceh Timur Aceh Besar Aceh Tamiang Aceh SelatanLangsa Nias Selatan District name 43 839 68 341 44 252 34 067 31 862 26 801 17 501 Population 7-12 years

Gayo Lues Sabang Lhokseumawe Aceh Barat Daya 11 586 19 478 16 314 3 048

Interpretation: Several districts reported to have almost all primary schools damaged or destroyed after the tsunami. This includes districts that were not directly affected by the tsunami but that could have been non-functional because of other reasons, i.e. the conflict. It is also questionable why Nias district reported 100% school damage whereas Nias Selatan reported 0%. By comparing primary school enrolment for children 7-12 years with school damage, there is no clear correlation between the extent of damage and the enrolment rate. The district with the lowest school enrolment is Aceh Jaya, with 88% of girls and 90% of boys enrolled for primary school. Also Aceh Barat had 90% enrolment for boys.

It is however questionable whether the indicator “school enrolment” actually reflects how many children go to school, as this number most likely derives from the number of children who sign up at the first day of school each year. To be able to draw conclusions on the true situation for the functionality of the school system it would therefore be more relevant to assess actual school attendance rates or school drop-out rates. Source file: BPS_SPAN_2005_NAD_by village.xls 30

Figure 27. Primary school enrolment for boys 7-12 years by sub-district for Aceh Barat district 79 80 82 82 85 86 90 92 94 95 97 % of boys 7-12 years attending school ) 100 hool (% hool Boys 7-12 yrs in school 50 Boys 7-12 yrs in school

Boys 7-12 years attending sc attending years 7-12 Boys 0 BUBON WOYLA JOHAN PAHLAWAN SAMA TIGA KAWAY XVI MEUREUBO Sub-district name 377 827 2 621 691 1 724 1 264 Population boys 7-12 years SUNGAI MAS WOYLA TIMUR ARONGAN LAMBALEK WOYLA BARAT PA NTE CEUREUMEN 268 321 716 497 654

Figure 28. Primary school enrolment for boys 7-12 years by sub-district for Aceh Barat district

Interpretation: By disaggregating enrolment rates, there appears to be significant differences in primary school enrolment across sub-districts. Aceh Barat, which had one of the lowest primary school enrolment rates for boys, have variations from 79% in Bubon sub-districts to 97% in Meureubo. It is therefore important to analyse school enrolment and school attendance rates by sub- district to be able to identify areas where these rates are particularly low. Source file: BPS_SPAN_2005_NAD_by village.xls

31 Discussion The conceptual framework can be a useful tool for analysis and illustration of TRIAMS indicators of impact and recovery in all the different sectors, including vital needs, basic social services, infrastructure and livelihoods. Even though the framework only is a first attempt and requires additional refinement, it can still be useful for planning of recovery efforts as well as refocusing of existing reconstruction activities. The framework will therefore be continuously updated and further developed based on new or improved data and on comments and suggestions from readers.

A main finding in this process was the poor quality of data collected for impact and monitoring purposes. A general quality issue was the fact that most available data sets lacked definitions of the indicators that were measured (e.g. what is implied by “severely affected village”) and metadata on sampling period and source were often missing. Many datasets had errors in data entry with miscoding of administrative levels, double entries and missing data. In other cases, two data sets reporting on the same indicator showed completely different results. For example measles immunization data differed XX percent when comparing data obtained from PHO (provincial Health Office) versus the data from a recent WHO survey of the same time period. Another example is the data set for “number of houses needed”, which had between 2-4 different entries for about 30% of the villages. We could only assume that these numbers should be added up and that the total yield would represent the total of numbers needed in each village, but it could also be that these numbers are representing different estimations of the overall need of the village and should hence be averaged. We sought to clarify some of these issues with different data managers but since data has been gathered from a number of different sources it was extremely difficult to find answers from the people responsible.

For impact data, the general problem was the lack of baseline information which made it practically impossible to draw conclusions on the extent of the damage and destruction following the tsunami. As an example, while school functionality obviously was very poor across many of the tsunami affected districts it was found to be equally poor in several districts not affected by the tsunami. Reasons include the impact of the conflict, which often was more severe in the inland districts. As no data was available for school functionality before the earthquake/tsunami, it was therefore not possible to use this indicator as an example of impact on basic social services.

For assessment of recovery it is often equally difficult to find good data for comparison. Often there is poor or no data of the estimated need of new units to be built/repaired (e.g. health facilities, fishing boats, houses etc) and progress can only be measured as new units built/repaired over time. Without a need indicator it is basically impossible to know when the target has been reached and when a sufficient number of units have been reconstructed or repaired. To overcome this issue, we had to make the assumption that the need equals the number of pre-tsunami units, e.g. the number of fishing boats that need to be rebuilt/repaired is the same as the number of boats that were destroyed/damaged by the tsunami. Although this would give an indication of how the recovery is progressing for one of the livelihood indicators, it is not optimal since many of the owners of the damaged boats died in the tsunami, hence there is no need to rebuild their boats.

Another issue with regards to setting targets for recovery efforts is the need for continuous monitoring and updating of the needs, as these changes with time as a result of e.g. population movements. As an example, the needs set for housing reconstruction derives from a survey

32 conducted in November 2005 (Garansi). However, the numbers of people who needed a house 14 months ago may not be the same number of people that need a house today. People may have solved their living situation in other ways, especially in areas where the housing reconstruction has been slow or have not yet started. Given that shelter is an essential necessity after a disaster and that housing needs is a “moving target”, it is crucial that household reconstruction is carefully monitored and that targets are updated in order to identify pockets of slow progress or inequitable distribution of new houses.

Conclusions, way forward and recommendations Data quality is key in trying to analyse and draw conclusions on recovery processes and gap analysis in a disaster situation. Questions must be raised at all levels on how to improve sampling of populations, data collection, data entry and other data management issues. There is an apparent need to harmonize datasets and generate metadata as a means of tracking data quality. Using a standardised code list for administrative divisions during data collection would have minimized errors in data entry. Finally, there is a great need to discuss the feasibility of the data to be collected for monitoring purposes. Many of the indicators in TRIAMS are indicator that would be interesting to monitor but that cannot be obtained at this stage. In some cases it would be better to go for data that is available rather than desirable.

Even though quality of data is far from optimal, it is not only a question of finding ways of improving data collection, but also of realising that certain data quality-trade-offs will be necessary. The reality is often extremely complex and human behaviour and movement is difficult to predict. This is something health policy makers need to accept and adjust to in the event of a disaster situation. Acknowledgement We are grateful to Neil Taylor and his team at the Information and Analysis Section (UNIMS) at UNORC, Banda Aceh for providing us with the data that enabled us to do the analysis in this document. We are also grateful to Hans Rosling at the Karolinska Institutet for good discussions on how to outline the analysis.

33 Annex 1 – the Karolinska Institutet Analytical Tool (KAT)

Objective for the KAT 1. To provide basic knowledge in regards to interpretation of data collected in Tsunami affected areas 2. To provide baseline data on tsunami impact/effects and recovery progress at sub-district level 3. To provide information on possible gaps in the recovery process 4. To provide guidance for the planning of tsunami recovery efforts at sub-district and district level

Is KAT useful? The objective of the relief efforts is to assist the affected population to return to normal life and to some extent address pre-tsunami inequities. Monitoring the progress of relief efforts are crucial in order to early identify areas with slow progress and re-focus accordingly. By analysing the allocation of resources, particularly in the social and health sectors, it is possible to draw conclusions on whether the aid has reached the poorest and neediest populations and whether it has corrected pre-tsunami inequities. Keeping this in mind, the purpose of the conceptual framework is to facilitate the analysis of data at the district and sub-district levels, using both absolute numbers and proportions in order to capture the variations and to allow for comparison across populations and geographical areas. The KAT is a simple tool to analyse and monitor tsunami related data (Figure 29). We are the first to admit that the KAT has its limitations as it is a pilot tool that needs to be continuously developed. We want it to be simple enough to decision makers with limited time available. We are grateful for inputs that can improve the tool. Annex 1 provides a step-by-step manual on how to use the KAT.

Figure 29. Karolinska Analytical Toolkit (KAT) with 2-field graph for impact and 4-field graph for recovery

34 Step by step manual

1. Copy the data that you want to illustrate and paste it into a separate excel file.

2. Decide whether the data you want to visualise is reflecting impact or progress. Impact data uses the pre-tsunami situation as baseline, i.e. population in 2004, number of health facilities before the tsunami etc. Progress data illustrates the recovery in relation to needs and targets. Impact data is best illustrated using the file with two fields in stacked columns. Progress data is best illustrated using the file with 4 fields in stacked columns (Figure 2).

3. Decide which variable that should reflect the affected population. This variable will thereby determine the area of the column.

4. Copy one column at the time from your clean data file. Begin with district names. Paste the information into the yellow marked area of the KAT form using “paste special” and “values” (right click on the mouse, select “paste special”, click on the option “values”. Click “ok”). Repeat the procedure for the other columns you want to insert.

5. Arrange the labels on the x-axis by moving the text boxes so that they can clearly be read. Use a line to indicate which text boxes connects with which bars in cases where it is crowded and the text box has to be moved further down.

35