CRVS Fellowship report Integrating verbal autopsy in routine mortality surveillance in Bangladesh

March 2019 Resources available from the University of CRVS technical guides Melbourne, Bloomberg Philanthropies Data Specific, technical and instructive resources in the form of for Health Initiative quick reference guides, user guides and action guides. These guides provide a succinct overview and/or instructions for CRVS course prospectuses the implementation or operation of a specific CRVS-related These resources outline the context, training approach, intervention or tool. course content and course objectives for the suite of CRVS trainings delivered through the Bloomberg Philanthropies CRVS tools Data for Health Initiative. Each course focuses on a specific Interactive and practical resources designed to influence CRVS intervention or concept, and is designed to support and align CRVS processes with established international or countries to strengthen their CRVS systems and data. best-practice standards. These resources, which are used extensively in the Initiative’s training courses, aim to change CRVS Fellowship reports and profiles practice and ensure countries benefit from such changes by The CRVS Fellowship Program aims to build technical developing critical CRVS capacity among technical officers capacity in both individuals and institutions to enhance and ministries. the quality, sustainability and health policy utility of CRVS systems in Fellows’ home countries. Fellowship reports Published by the University of Melbourne, Civil Registration are written by Fellows as a component of the program, and Vital Statistics Improvement, Bloomberg Philanthropies and document, in detail, the research outcomes of their Data for Health Initiative. Fellowship. Fellowship profiles provide a summary of Melbourne School of Population and Global Health Fellows’ country context in relation to CRVS, an overview Building 379 of the Fellowship experiences, the research topic and the 207 Bouverie Street projected impact of findings. Carlton, VIC 3053 CRVS analyses and evaluations Australia These analytical and evaluative resources, generated through [email protected] the Initiative, form a concise and accessible knowledge-base www.mspgh.unimelb.edu.au/dataforhealth of outcomes and lessons learnt from CRVS initiatives and interventions. They report on works in progress, particularly for large or complex technical initiatives, and on specific Made possible through funding from components of projects that may be of more immediate Bloomberg Philanthropies relevance to stakeholders. These resources have a strong www.bloomberg.org empirical focus, and are intended to provide evidence to assist planning and monitoring of in-country CRVS technical initiatives and other projects Author CRVS best-practice and advocacy Generated through the Initiative, CRVS best-practice and Mr Saiful Islam, Ministry of Health and Family Welfare, advocacy resources are based on a combination of technical Bangladesh. knowledge, country experiences and scientific literature. These resources are intended to stimulate debate and ideas for in-country CRVS policy, planning, and capacity building, Suggested citation and promote the adoption of best-practice to strengthen Islam, S. Fellowship report: Integrating verbal autopsy in CRVS systems worldwide. routine mortality surveillance in Bangladesh. CRVS Fellowship CRVS country reports reports and profiles. Melbourne, Australia: Bloomberg CRVS country reports describe the capacity-building Philanthropies Data for Health Initiative, Civil Registration and experiences and successes of strengthening CRVS systems Vital Statistics Improvement, University of Melbourne; 2018. in partner countries. These resources describe the state of CRVS systems-improvement and lessons learnt, and provide a baseline for comparison over time and between countries. CRVS Fellowship reports and profiles 3 6 6 7 9 4 5 6 8 13 14 15 15 17 21 27 31 33 34 35 37 42 43 43 48 48 48 10 15 31 35 40 44 45 48 ...... Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 in Bangladesh | Version autopsy in routine mortality surveillance Integrating verbal report: Fellowship ...... process...... autopsy interview Verbal ...... analysis...... Post-interview Ethical considerations autopsies conducted...... Characteristics of verbal ...... Causes of death...... in adult mortality...... Patterns ...... in child mortality...... Patterns estimates...... Comparison with Global Burden of Disease System data...... Comparison with Health and Demographic Surveillance ...... Undetermined cases...... autopsy supervisors...... Verbal ...... autopsy interviewers Verbal Bangladesh...... autopsy in Common problems affecting verbal in Bangladesh...... SmartVA Recommendations to improve ...... Study limitations...... Library...... Gateway: Knowledge CRVS of Melbourne, D4H Initiative, University Learning Centre...... Gateway: Knowledge CRVS of Melbourne, D4H Initiative, University Courses...... Gateway: Knowledge CRVS of Melbourne, D4H Initiative, University Methods. Results...... steps...... Discussion and next ...... project in Bangladesh SmartVA ...... autopsy data analysis of verbal Results: Quantitative ...... SmartVA through data generated Results: Plausibility of cause of death study...... Results: Qualitative ...... Discussion...... implementation areas 1 SmartVA Annex Annex 2 Statistical tables...... Annex ...... and products Related resources ...... Acknowledgements Contents Acronyms and abbreviations and Acronyms ...... Summary...... Introduction ...... Methods...... 4 CRVS Fellowship reports and profiles Fellowship report: Integratingverbal autopsy inroutine mortalitysurveillance inBangladesh |Version 1319-01 been possible. Finally, Iwould like tothankalltheparticipantsandcommunitypeople,withoutwhomthisstudywould nothave Ministry ofHealthandFamily Welfare andDirectorateGeneralofHealthServices(Bangladesh). Burden ofDiseaseGroup,MelbourneSchoolPopulation andGlobalHealth,University ofMelbourne,Australia; andthe the verbal autopsy datafromthecommunityandthuscontributedtoresearch.This studywas supported bytheGlobal I would alsolike toacknowledge thededicationof, andtheeffortsmadeby, communityfield workers whohave collected Bangladesh. especially UH&FPO,Kaliganj;BangladeshBureauofStatisticsandInternationalCentreforDiarrhoealDiseaseResearch, Civil Surgeon,UpazilaNirbahiOfficer, UpazilaHealthand Family PlanningOfficer(UH&FPO)oftheirrespective upazila, and DrShahAliAkberAshrafiMrMoyeen Uddin,CountryCoordinator, BloombergDataforHealthInitiative, Bangladesh. I would like toshow myheartfeltgratitudetoProfAbulKalamAzad,DirectorGeneral,DirectorateGeneralofHealthServices; Health Services;Policy Advisor, a2i;CountryCoordinator, BloombergDataforHealthInitiative, Bangladesh. System Improvement inBangladesh,CabinetDivision;Director, ManagementInformationSystem,DirectorateGeneralof Secretary, CoordinationandReforms,CabinetDivision;AdditionalSecretaryProjectDirector, Technical SupportforCRVS implementing SmartVA. Ialsoexpress gratitudetoSecretary, HealthServicesDivision,MinistryofandFamily Welfare; tremendous supportandcommitmenttoimprove thecivilregistrationandvitalstatistics(CRVS) systeminBangladeshby McLaughlin, DeputyDirectoroftheGlobalBurdenDiseaseGroup;DrTim Adair;andDrMdHafizurChowdhury fortheir acknowledge LaureateProfessorAlanLopez,DirectoroftheGlobalBurdenDiseaseGroup;AssociateProfDeirdre enough toallocatehervaluabletimeprovide technicalsupport,motivatingthoughtandencouragement.Igratefully and theabilitytocompletetask.SpecialthankswithhonourmysupervisorDrGulshanAraKhanom,whowas kind First andforemost,withheartfeltdevotion, IamgratefultoalmightyAllahforblessingmewithsuchopportunitylearning Acknowledgements CRVS Fellowship reports and profiles 5 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 in Bangladesh | Version autopsy in routine mortality surveillance Integrating verbal report: Fellowship cause of death vital statistics civil registration and fraction case specific mortality Data for Health Global Burden of Disease System Health and Demographic Surveillance Research, Bangladesh International Centre for Diarrhoeal Disease identification information technology registration local government noncommunicable disease Open Data Kit Health Metrics Research Consortium Population autopsy verbal CRVS CRVS CSMF D4H GBD HDSS icddr,b ID IT LGR NCD ODK PHMRC VA COD Acronyms and abbreviations Acronyms Summary

Death registration is low in Bangladesh, where many deaths occur in the community, and mostly at home. Since these deaths occur outside of the formal healthcare system, it is difficult to ascertain the cause of death (COD). In these situations, verbal autopsy (VA) can be a valuable practical tool to determine COD. To overcome data gaps for CODs, the Government of Bangladesh has introduced VA, using the SmartVA instrument developed by the University of Washington. Supported by the Bloomberg Data for Health (D4H) Initiative and the University of Melbourne, an initial pilot project in Kaliganj Upazila (sub-district) was extended to 13 other upazilas. VA is implemented as part of broader efforts to strengthen civil registration and vital statistics (CRVS) and improve the availability and quality of data on COD in Bangladesh. As a result, all deaths identified in the community are officially registered before conducting a VA.

Methods

This report has three major aims:

1. To describe COD patterns in Bangladesh using data from quantitative VA 2. To assess the plausibility of these CODs by comparing them with other sources of COD data available in Bangladesh 3. To describe the experiences of VA interviewers and supervisors, and the challenges faced in conducting VA interviews.

The work for the first two aims was carried out from March 2017 to January 2018 in 10 upazilas within five districts of Bangladesh. During existing routine household visits, community health workers identified any deaths that had occurred. For each death identified, the community health worker notified the local civil registration authorities and encouraged the family to follow up so that the death was registered. A death certificate was then issued.

For the VA, a health assistant interviewed someone who was familiar with the condition of the deceased person – usually a spouse, mother or another close family member – at their home. The health assistant was trained to use a SmartVA questionnaire on a tablet (also known as the shortened Population Health Metrics Research Consortium [PHMRC] questionnaire).

From March 2017 to January 2018, 7837 SmartVA interviews were conducted on deaths from 7424 adults, 239 children and 174 neonates. These VA interview data were analysed using SmartVA-Analyze (a computer-based automated method for assigning CODs) to determine the CODs. The findings were summarised and the plausibility of leading CODs was assessed by comparing the COD data with data from Global Burden of Disease (GBD), Matlab Health and Demographic Surveillance System (HDSS) and other sources.

For the third aim, a qualitative study was conducted only in Kaliganj. A semistructured questionnaire was used to collect qualitative data from 5 supervisors and 16 interviewers on the experiences and challenges of conducting VA. The data were summarised and grouped into themes.

Results

SmartVA appears to be effective in generating CODs from community deaths. An improvement in death registration coverage was also seen. The number of VAs was higher for people older than 60 years, and higher for males than females in all age groups. Deaths were from noncommunicable diseases (79.9 per cent), communicable diseases (9.5 per cent) and injuries (10.6 per cent). In adults, the top three CODs were ischaemic heart disease (22 per cent), stroke (19 per cent) and chronic respiratory disease (17 per cent). Drowning was the leading COD in children (26 per cent) and pneumonia was the leading COD in neonates (30 per cent). Undetermined CODs were found in 18 per cent of cases, mostly in decedents aged over 60. Reasons for the undetermined CODs might include: CRVS Fellowship reports and profiles Fellowship CRVS

6 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 ■ Insufficient information collected to assign a COD ■ Poor quality VA interview ■ Inappropriate understanding of the medical terms used during interview.

The qualitative study revealed challenges such as:

■ Duplication of death registrations ■ Lack of timely support from the local registration office in getting a VA identification number ■ Omission of VA activities from the job description of the health worker ■ Difficulties in asking the interviewee about HIV/AIDs ■ Lack of a reporting system for supervision and monitoring at the field level ■ Lack of reliable IT ■ Lack of community awareness about VA.

Discussion and next steps

This study showed that VA is a feasible option for determining COD patterns in rural populations in Bangladesh, where most deaths occur outside formal healthcare settings.

Noncommunicable diseases are the leading CODs in adults, with stroke, ischaemic heart disease and chronic respiratory disease comprising the top three. Community awareness and health programs targeting noncommunicable diseases are important to prevent premature deaths from such diseases.

To improve VA implementation and scaleup SmartVA in Bangladesh, regular coordination and support from the relevant health department, local registry officials and other CRVS stakeholders is crucial. Since most deaths occur at home, it would be cost- effective if the government could generate population-level COD data from VA on a nationally representative sample of deaths. Establishing an automated system to flag any repeated death registrations and generate unique VA identifications for each individual case would help minimise errors due to duplication.VA-related activities should be included in the job description of the community health workers to make VA routine in mortality surveillance. On-the-job training and refresher courses for VA interviewers and supervisors to maintain the quality of the interview would also help improve the quality of VAs. A proper monitoring and evaluation framework is critically important for appropriate implementation of VA, which is needed to generate high-quality COD data. Technical support services, such as IT support, should be ready to act at the national and local levels to resolve quickly any issues related to VA implementation. Improving public awareness about the importance of COD and the role of VA would facilitate the nationwide scale up of VA in Bangladesh. CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 7 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh

Introduction

Cause-specific mortality statistics by age and sex are important for public health decision- making, planning and monitoring. Even so, globally, two-thirds of all deaths are not counted, especially in low-income countries.1 Only 14 of the 75 low and middle-income countries report on death registrations. In these countries, most of the deaths occur at home, where a physician or other trained person are not available to certify the cause of death (COD). Ultimately, these deaths are not registered.2

Bangladesh is one of the low-income countries in South-East Asia where most of the Most deaths in deaths occur in the community, and birth and death registrations are low. Bangladesh has a Bangladesh occur in the population of about 165 million, and an estimated 900 000 people die every year. Only about community, and very few 100 000 of these deaths – in general, those that occur in public hospitals – are reported to are registered. the Ministry of Health and Family Welfare (MOHFW).3 This means that COD information is not available most of the deaths in the country. In addition to hospital data, sample survey and some other periodical household surveys are being carried out to identify CODs in Bangladesh, but major gaps in, and issues with collecting, COD data remain.

The Bangladesh Sample Vital Registration System (SVRS) is a sample enumeration system – it does not register actual deaths. CODs are ascertained through a lay reporting system by SVRS staff (casual registrars who are non-medical staff), who use a checklist of 64 specific causes and an open space for other causes. For simplicity, these 64 causes are grouped into 26 classes. Because SVRS staff are not medically qualified or trained in any verbal autopsy (VA) method, they cannot reliably ascertain CODs. As a result, COD data from the SVRS are unreliable and are not used by the government for any policy or reporting purposes.4

The National Institute of Population Research and Training (NIPORT) and the MOHFW conduct the nationwide Demographic Health Survey about every 3 years. The International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b) and another the Inner City Fund assist with the survey. VA is only used to ascertain COD for children younger than five years.5

There are several sources NIPORT (with technical support from icddr,b) has also conducted the three nation-wide of cause of death data Bangladesh Maternal Mortality Surveys and administered VA for all deaths of women of available in the country. childbearing age. This survey generates indicators related only to maternal healthcare services, maternal mortality ratios and causes of maternal deaths (using VA) for Bangladesh.6

1 World Bank, WHO. Global civil registration and vital statistics scaling up investment plan 2015–2024. New York, USA: World Bank; 2014. 2 de Savigny D et al. Integrating community-based verbal autopsy into civil registration and vital statistics (CRVS): system-level considerations. Global Health Action.2017; (10):1272882. 3 Azad PD. Community and hospital based death registration with COD – progress of Data for Health Initiative in Bangladesh. Mohakhali, Dhaka, Bangladesh: Directorate General of Health Services; 2018. 4 Bangladesh Bureau of Statistics. Report on Bangladesh sample vital statistics. Dhaka, Bangladesh: Bangladesh Bureau of Statistics; 2017. 5 National Institute of Population Research and Training. Bangladesh Demography and Health Survey. Dhaka, Bangladesh: Ministry of Health and Family Welfare; 2014. 6 National Institute of Population Research and Training et al. Bangladesh Maternal Mortality and Health Care Survey 2010. Dhaka,

CRVS Fellowship reports and profiles Fellowship CRVS Bangladesh: National Institute of Population Research and Training, MEASURE Evaluation, icddr,b; 2012.

8 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Another source of population-level COD data is the Health and Demographic Surveillance System (HDSS). icddr,b assists the HDSS site in Matlab to carry out research and evaluation activities. The Matlab HDSS site maintains a continuous demographic surveillance system. When a death is identified in Matlab’s surveillance population, a medical assistant administers a VA interview at the household level and assigns a COD. Matlab’s surveillance population, however, is only about 234 000 people, all in the Chittagong Division.7

Therefore, the country needs to increase the number of birth and death registrations, and determine CODs in a community that represents the general population. The method must be minimum cost, take little time and be easy to use in rural and remote areas. In 2016, the Government of Bangladesh started a civil registration and vital statistics (CRVS) project to improve birth and death registrations, COD identifications and analysis of COD data. The Bloomberg Philanthropies Data for Health (D4H) Initiative and the University of Melbourne support the project.

Verbal autopsies begin In the absence of medical certification, VA is the only available tool to identify the probable with a structured COD.8 VAs work by interviewing someone close to the decedent to determine their interview. This data is symptoms before death. This information, in combination with demographic data, is then then processed by an used to assign a probable COD. Verbal autopsy is one of the Bloomberg D4H Initiative’s algorithm to assign a interventions to improve death registration and identify CODs in communities. An automated likely cause of death. diagnostic algorithm that uses age, sex and geographical location, along with an International Classification of Disease version 10 code, assigns CODs.9

SmartVA project in Bangladesh

In January 2017, Kaliganj Upazila (sub-district), within Gazipur District, became the first to pilot SmartVA data collection. Since then, VA has been scaled up in 13 upazilas in 8 divisions (see Annex 1):

■ Kaliganj, Kaliakoir, Kapasia, Sreepur and Sadar upazilas in Gazipur District ■ Trishal and Bhaluka upazilas in Mymensingh District ■ Phultala Upazila in Khulna District ■ Bishwanath Upazila in Sylhet District ■ Kishoreganj Upazila in Nilphamari District ■ Paba Upazila in Rajshahi District ■ Gournadi Upazila in Barishal District ■ Anowara Upazila in Chittagong District.

The D4H Initiative supported the program in the first upazilas in Gazipur and Mymensingh

districts. The remainder are supported by the Government of Bangladesh. CRVS Fellowship reports and profiles

Table 1 describes some characteristics of the 13 upazilas.

7 Mahfuzur R et al. Health and demographic surveillance system. Volume fifty. Registration of health and demographic events 2015. Dhaka, Bangladesh: icddr,b; 2017. 8 de Savigny D et al. Integrating community-based verbal autopsy into civil registration and vital statistics (CRVS): system-level considerations. Global Health Action.2017; (10):1272882. 9 University of Melbourne. Automated VA. Available at: https://crvsgateway.info/Automated-VA~557(accessed 1 February 2019).

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 9 Table 1: Basic characteristics of SmartVA implementation areas

CDR Population Master District Upazila Urban population (2017) (2011) trainers Gazipur 4.2 Kaliganj 265 276 17 Jan 2017

Kaliakoir 483 308 33 Oct 2017 Kapasia 342 162 4 Oct 2017 Sreepur 492 792 26 Oct 2017 Sadar 1 820 374 38 Sep 2017 Mymensingh 5.0 Trishal 419 308 14 Oct 2017 Bhaluka 430 320 15 Dec 2017 Sylhet 5.1 Bishwanath 232 573 – Dec 2017 Khulna 4.6 Phultala 83 881 38 Dec 2017 Nilphamari 5.9 Kishoreganj 261 069 – Jan 2018 Rajshahi 5.0 Paba 314 196 28 Feb 2018 Barishal 5.6 Gournadi 188 586 23 Mar 2018 Chittagong 4.1 Anowara 259 022 2 May 2018

– = data not available; CDR = crude death rate; VA = verbal autopsy Source: Population and Housing Census 2011

All the upazilas implemented the model used in the Kaliganj pilot (see Methods). This process is more cost-effective and faster, and provides standardised results across populations when compared with physician-certified VA.10

The government is planning to integrate SmartVA within the District Health Information System under the Director General of Health Services, so the COD data can be stored with other sources of data. The data can then be shared with the national CRVS system to produce national mortality statistics and design public health policies and interventions to prevent premature deaths.

Methods

To further expand the SmartVA project in Bangladesh, it is important to understand if the distribution of causes of death (CODs) already reported is plausible. Learning about the experiences of the verbal autopsy (VA) interviewers and supervisors, and how any challenges can be improved, is also useful. Unfortunately, until now this information has not been captured. Therefore, this study examines CODs in the community, the plausibility of the COD distribution, experiences of VA interviewers and supervisors, and challenges and possible solutions to improve SmartVA implementation in Bangladesh.

10 Flaxman A et al. Collecting verbal autopsies: improving and streamlining data collection processes using electronic tablets.

CRVS Fellowship reports and profiles Fellowship CRVS Population Health Metrics. 2018; 16:3.

10 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Aims The study aimed to:

■ Describe the COD patterns for community deaths in selected areas of Bangladesh ■ Compare the results with other sources of COD data available in Bangladesh ■ Describe the experiences of VA interviewers and supervisors, and challenges faced during the VA interview.

Study period The study period took place from March to June 2018.

The data collection was from February 2018 to March 2018.

Study area Quantitative VA data were collected from 10 upazilas:

■ Kaliganj, Kaliakoir, Kapasia, Sreepur and Sadar upazilas in Gazipur District ■ Trishal and Bhaluka upazilas in Mymensingh District ■ Phultala Upazila in Khulna District ■ Bishwanath Upazila in Sylhet District ■ Kishoreganj Upazila in Nilphamari District.

The last three upazilas(Paba, Gournadi and Anowara)were not included, because they just began VA during the study period, and thus had no data to provide.

Qualitative data were collected from Kaliganj, because it was the pilot upazila and other upazilas adopted the model used there.

Study population and size The 10 upazilas used in the study had a total population of 4 831 063, providing 7842 VA interviews. The quantitative study included only community deaths, thus excluding hospital deaths, Bangladeshi deaths that occurred in another country, police cases and stillbirths.

For the qualitative component, 5 VA interviewers and 16 VA supervisors in Kaliganj were selected to complete a questionnaire.

Of the five supervisors, one was a health inspector and four were assistant health inspectors (three men and two women).

All 16 VA interviewers were health assistants – community-level health workers with non-

medical backgrounds (nine men and seven women). CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 11 Training Before starting data collection, SmartVA training was conducted using a cascade approach. Training of trainers was applied to train master trainers, who then trained VA interviewers and supervisors in selected upazilas.

The training of trainers used a 5-day course, and subsequent interviewer training was for 4 days. The University of Melbourne facilitated the training of trainers using standardised training guidelines and manuals. At least one interviewer was trained for each ward, and at least one supervisor was trained for each union for all 10 upazilas.11

In total, training covered:

■ 70 master trainers (consultants, medical officers, upazila family planning officers, programmers, IT consultants, management information system statisticians and the Directorate General of Health Services)

■ 100 supervisors (health inspectors, assistant health inspectors, family planning inspectors and sanitary inspectors)

■ 500 interviewers (health assistants and family welfare assistants).

All groups were trained in using SmartVA on a tablet, supervision techniques, and ethical and sensitivity issues related to conducting VAs.

In addition, 20 IT people underwent 2 days of training about IT and VA data management. Trainees included management health information system medical officers, IT consultants, programmers, statisticians and IT stakeholders.

Tools For the quantitative data collection, we used a shortened Population Health Metrics Research Consortium (PHMRC) questionnaire, adopted from the Institute of Health Metrics and Evaluation, in Excel format. This was translated into local language, tested at the Matlab Health and Demographic Surveillance site, reviewed, and finally translated into an XML format suitable for use by the Open Data Kit (ODK) software installed the mobile tablets.

Using the PHMRC questionnaire and ODK, the trained VA interviewer collected information on the decedents’ symptoms before death. The data were then downloaded from the ODK Aggregate server and analysed using SmartVA-Analyze (Tariff 2.0).

The qualitative data were collected using a semistructured questionnaire. The VA supervisors were asked questions about):

■ The number of VA interviews they have observed ■ Their use of the supervision checklist ■ The amount of data discarded because of refusal to answer, double entries and so on ■ Common IT errors (eg with the tablet) while entering information, and how such errors were resolved

■ Debriefing sessions and monthly meetings ■ What worked well, and challenges and how they were addressed ■ How VA implementation can be improved across the country.

11 A ward is an administrative division of a city or union that typically elects, and is represented by, a councillor(s). Union is the smallest rural administrative and local government units in Bangladesh. Each union is made up of nine wards. Usually one

CRVS Fellowship reports and profiles Fellowship CRVS village is designated as a ward.

12 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 For VA interviewers, the questionnaire included questions about):

■ How deaths were identified for VA ■ How VA interviews were set up ■ The number of refusals for VA interviews and why ■ How interviewers travelled to the interviews ■ Whether VA was part of their normal work schedule ■ Who was interviewed and where ■ How long the interviews took to complete ■ Difficulties encountered while asking questions and getting responses, and how they were managed

■ Problems encountered when recording responses on the tablets ■ Whether all VAs were connected to death notifications/registrations at the local registry office

■ Factors facilitating or limiting obtaining a registration number for conducting the VA ■ What worked well in the field, and challenges and how they were managed.

The VA supervisors and interviewers completed the questionnaire on paper. A separate group discussion was held with the VA supervisors and interviewers to discuss any questions they did not understand.

Verbal autopsy interview process

Figure 1 is a flow chart of the VA process, including how deaths were identified in the community.

At the first step, the VA interviewer (health assistant or family welfare assistant) learned of a death during a regular home visit. Health assistants visited households in a certain area once a month, and family welfare assistants visited homes about every 45 days.

After the VA interview, the interviewer filled out the death registration form at the decedent’s home. They then submitted the form to the union supervisor (eg health inspector, assistant health inspector, family planning inspector). The supervisors met once or twice a month at a community clinic, union health and family welfare centre or union subcentre. They checked the notification form, identified the duplicated form and then submitted the form to the local government registration (LGR) office. The LGR office entered the information from the forms online and supplied a 17-digit VA identification number to the supervisor, which is passed to the respective VA interviewer. CRVS Fellowship reports and profiles

VAs were generally Next, the VA interviewer visited the decedent’s home to set the date and time of the VA conducted between 6 interview. This was after an adequate mourning period, usually between 6 weeks and 3 weeks and 3 months months after the death, up to a maximum of 1 year, to avoid recall bias. Religious and after the death. private occasions were avoided. The interviewer also selected the best person to provide the information on signs, symptoms and associated illnesses before death. The interviewer asked the family to have some of the decedent’s documents ready, such as their national identification card and medical records.

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 13 The interviewer then visited the deceased person’s house to conduct the VA interview. The interviewer explained the purpose of their visit and the importance of VA, and asked for permission to start the interview. After getting verbal consent, the interviewer started the VA interview. A VA supervisor accompanied every fifth case and checked the tablet after the interview.

Finally, the interviewer sent the form to the ODK server.

Figure 1 Smart VA process in Bangladesh

AHI= assistant health inspector; BRIS = Birth Registration Information System; CC = community clinic; CRVS = civil registration and vital statistics; CSV = comma-separated variables; DGHS = Directorate General of Health Services; FPI = family planning inspector; FWA = family welfare assistant; FWC = family welfare clinic; HA = health assistant; ID = identification; ODK= Open Data Kit; VA = verbal autopsy.

Post-interview analysis

The COD data were disaggregated by age, sex and geography using SmartVA-Analyze (version 2) and Microsoft Excel.

SmartVA-Analyze assessed the plausibility of the COD distribution by comparing it with other data sources (eg Global Burden of Disease, and the Matlab Health and Demographic Surveillance System). The CODs in the other data were grouped and matched with SmartVA CODs in Microsoft Excel. Any CODs that could not be matched to either source were recorded using the original name.

The qualitative information was entered into the Excel spreadsheet, then summarised and grouped thematically. CRVS Fellowship reports and profiles Fellowship CRVS

14 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Ethical considerations

All participants made informed verbal consent before the VA and qualitative data collection. Participation was voluntary, and the participants had the right to refuse or withdraw at any time during data collection. All the information was treated as confidential and only used for this study. Participants were not at any risk and did not receive compensation for taking part in this study.

Results: Quantitative analysis of verbal autopsy data

Almost 8000 verbal From March 2017 to January 2018, 7837 verbal autopsies (VAs) were conducted in the study autopsies were area, comprising 7425 adults, 239 children and 174 neonates. The deaths were mostly at conducted during the home (N = 6947, 89 per cent), followed by in a hospital or other health facility (N = 496, 6 per study period. cent), brought in dead (N = 287, 4 per cent) and other places.

Characteristics of verbal autopsies conducted

By district The number of VAs conducted in Gazipur District was high in comparison with Mymensingh, Sylhet, Khulna and Nilphamari districts because Gazipur District was the site of the pilot VA project in Bangladesh. Table 2 illustrates the number of VA interviews in the five districts.

Table 2 Number of verbal autopsies (VAs) conducted, by district and start date

Number of Start date of District VAs VA Gazipur 5356 Jan 2017 Mymensingh 1738 Oct 2017 Sylhet 328 Dec 2017 Khulna 321 Dec 2017 Nilphamari 7 Jan 2018 CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 15 By upazila Table 3 illustrates the number of VAs conducted in each upazila.

In Kaliganj Upazila, 1387 VAs were conducted out of 1655 registered deaths – that is, 84 per cent of registered deaths had an associated VA. A separate study estimated that registration completeness in Kaliganj from March 2017 to January 2018 was 92 per cent, meaning that nearly all deaths in the period were registered within 1 year.

Table 3 Number of verbal autopsies (VAs) conducted, by sub-district and start date

Number of Start date of Upazila (sub-district) VAs VA Kaliganj 1387 Jan 2017 Sadar 382 Sep 2017 Kaliakoir 876 Oct 2017 Kapasia 1287 Oct 2017 Sreepur 1423 Oct 2017 Trishal 1416 Oct 2017 Bhaluka 323 Dec 2017 Bishwanath 328 Dec 2017 Phultala 309 Dec 2017 Kishoreganj 7 Jan 2018

Age and sex of deceased Most VAs were for people 60 years and older. All age groups contained more men than women (Figure 2).

Figure 2 Number of verbal autopsies conducted, by age group and sex

Figure 3 gives the distribution of VAs conducted for children less than 5 years old. CRVS Fellowship reports and profiles Fellowship CRVS

16 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Figure 3: Number of verbal autopsies conducted in children, by age group and sex

Causes of death

Broad disease categories Overall, more deaths were caused by noncommunicable diseases (79.9%) than by communicable diseases (9.5%) and injuries (10.6%).

Of the deaths from noncommunicable diseases:

■ 41 per cent were caused by cardiovascular disease ■ 32 per cent were caused by other noncommunicable diseases (chronic respiratory disease, cirrhosis, diabetes, renal failure, digestive disease, other defined cause of child deaths and other noncommunicable disease)

■ 7 per cent were caused by cancer. CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 17 Broad disease categories by age group and sex Age-disaggregated cause of death (COD) data show that most infants (<1 year) died from communicable diseases, whereas noncommunicable disease deaths became more prevalent in adulthood (Figure 4).

Figure 4 Proportion of causes of death, by age group

Group 1 = communicable, maternal, neonatal and nutritional conditions; Group 2 = noncommunicable diseases; Group 3 = injuries and external causes of death

Table 4 shows the distribution of CODs by age group and sex. CRVS Fellowship reports and profiles Fellowship CRVS

18 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Table 4 Causes of death, by age and sex

Deaths Age group Cause of death Male Female Total Neonates (0–28 Communicable 46 44 90 days) diseases Maternal, perinatal 49 18 67 and nutritional diseases Noncommunicable 0 0 0 diseases Injuries 0 0 0 Undetermined 9 8 17 Subtotal 104 70 174 Children (29 days – Communicable 30 24 54 11 years) diseases Maternal, perinatal 0 0 0 and nutritional diseases Noncommunicable 46 23 69 diseases Injuries 37 31 68 Undetermined 24 24 48 Subtotal 137 102 239 Adults (12 years Communicable 162 96 258 and over) diseases Maternal, perinatal 0 51 51 and nutritional diseases Noncommunicable 3375 1862 5237 diseases Injuries 371 135 506 Undetermined 783 589 1372 Subtotal 4691 2733 7424 Total 4932 2905 7837

Leading causes of death by upazila Table 5 shows the three most common CODs for each upazila, as well as the proportion of ‘undetermined’ deaths (full results are in Annex 2, Table 9). CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 19 Table 5 Completeness of additional information

Leading COD Second-highest Third-highest Undetermined’ Upazila (%) COD (%) COD (%) COD (%) Kaliganj Stroke IHD CRD 10 (23.7) 20.6 (17.4) Kaliakoir IHD CRD Stroke 22 (20.3) 15.8 (12.3) Kapasia Stroke CRD IHD 24 (18.4) (15.3) (15.3) Sreepur IHD Stroke CRD 16 (21.2) (17.6) (13.8) Sadar, Gazipur IHD Stroke Chronic resp. 22 (20.1) (14.4) (11.1) Trishal CRD IHD Stroke 18 (16.2) (15.6) (13.2) Bhaluka IHD Stroke CRD 21 (25.9) (12.8) (11.2) Bishwanath IHD CRD Stroke 22 (31.3) (21.9) (9.0) Phultala IHD Stroke CRD 19 (18.5) (14.3) (9.0) Kishoreganj IHD Stroke RTA 0 (28.5) (28.5) (28.5)

COD = cause of death; CRD = chronic respiratory disease; IHD = ischaemic heart disease; RTA = road traffic accident CRVS Fellowship reports and profiles Fellowship CRVS

20 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Patterns in adult mortality

The leading CODs in adults (undistributed and redistributed) were analysed using SmartVA- Analyze (Table 6). SmartVA-Analyze can redistribute the undetermined cases within 34 CODs for adults, 22 CODs for children and 7 CODs for neonates, based on the probability estimates in a country.

Table 6 Top 20 causes of death, adults, before and after redistribution of undetermined

SmartVA SmartVA COD redistributed undistributed (%) (%) This study found a lower Ischaemic heart disease 19 22 rate in death certification errors compared to Undetermined 18 0 other, similar studies. Stroke 16 19 Chronic respiratory disease 15 17 Diabetes 4 5 Renal failure 3 4 Cirrhosis 3 3 Falls 2 2 Road traffic accident 2 3 Other noncommunicable disease 2 4 Pneumonia 1 2 Lung cancer 1 2 Oesophageal cancer 1 1 Prostate cancer 1 1 Leukaemia/lymphoma 1 1 Tuberculosis 1 1 Other injury 1 0 Maternal-related death 1 1 Other cancer 1 1 Breast cancer 1 0 Other infectious disease 0 1 Suicide 0 1 Drowning 0 1 CRVS Fellowship reports and profiles Cause-specific mortality fractions for adult males (>12 years)

Most deaths were In total, 4932 male VAs were conducted. The main CODs were: due to chronic diseases (NCDs). ■ Ischaemic heart disease (24.1 per cent, N = 1032) ■ Chronic respiratory disease (17.4 per cent, N = 759) ■ Stroke (17.3 per cent, N = 702) ■ Diabetes (4.8 per cent, N = 191) ■ Road traffic accident (3.9 per cent, N = 150).

The distribution of adult male cause-specific mortality fractions (CSMFs) is shown in Figure 5.

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 21 Figure 5 Cause-specific mortality fractions for adult males,>12 years

Cause-specific mortality fractions for adult female (>12 years) In total, 2905 female VAs were conducted. The main CODs were:

■ Stroke (22.4 per cent, N = 530) ■ Ischaemic heart disease (18 per cent, N = 418) ■ Chronic respiratory disease (14.9 per cent, N = 363) ■ Diabetes (6.1 per cent, N = 142) ■ Noncommunicable disease (4.2 per cent, N = 55).

The distribution of adult female CSMFs is shown in Figure 6. CRVS Fellowship reports and profiles Fellowship CRVS

22 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Figure 6: Cause-specific mortality fractions for adult females, >12 years

Ischaemic heart disease The number of deaths from ischaemic heart disease was 1450. Figure 7 shows that the number of deaths from ischaemic heart disease was highest in adults aged 60–69 years (25 per cent, N = 356) and 70–79 years (22 per cent, N = 321). CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 23 Figure 7: Age–sex distribution of ischaemic heart disease deaths in adults, >12 years

Stroke The number of deaths from stroke was 1232. Figure 8 shows that the number of deaths from stroke was highest in adults aged 60–69 years (23 per cent, N = 286) and 70–79 years (23 per cent, N = 286).

Figure 8: Age–sex distribution of stroke deaths in adults, >12 years CRVS Fellowship reports and profiles Fellowship CRVS

24 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Chronic respiratory diseases The number of deaths from chronic respiratory disease was 1122. Figure 9 shows that the number of deaths from chronic respiratory disease was highest in adults aged 70–79 years (30 per cent, N = 336) and >80 years (32 per cent, N = 359).

Figure 9 Age–sex distribution of chronic respiratory disease deaths in adults, >12 years

Diabetes The number of deaths due to diabetes was 333. Figure 10 shows that the number of deaths from diabetes was highest in the 60–69-yearage group (24 per cent, N = 79) and the 70–79- year age group (22 per cent, N = 73).

Figure 10 Age–sex distribution of diabetes deaths in adults, >12 years CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 25 Cirrhosis The number of deaths from cirrhosis was 190. Figure 11 shows that the number of deaths from cirrhosis was highest in adults aged 50–59 years (23 per cent, N = 44).

Figure 11 Age–sex distribution of cirrhosis deaths in adults, >12 years

Renal failure The number of deaths from renal failure was 232. Figure 12 shows that the number of deaths from renal failure was highest in adults aged 40–49 years (20 per cent, N = 47) and 60–69 years (20 per cent, N = 48).

Figure 12 Age–sex distribution of renal failure deaths in adults, >12 years CRVS Fellowship reports and profiles Fellowship CRVS

26 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Road traffic accidents The number of deaths from road traffic accidents was 166. Figure 13 shows that the number of deaths from road traffic accidents is highest in adults aged 30–39 years (21 per cent, N = 35).

Men are more likely to die in a road traffic accident than women (90 per cent of road traffic deaths versus 10 per cent).

Figure 13 Age–sex distribution of road traffic accident deaths in adults, >12 years

Patterns in child mortality

Cause-specific mortality fraction for neonates (0 to 28 days) In total, 174 neonate VAs were conducted. The main CODs were:

■ Pneumonia (30 per cent) ■ Meningitis/sepsis (25 per cent) ■ Birth asphyxia (20 per cent) ■ Preterm delivery (15 per cent) ■ Congenital malformation (10 per cent).

The distribution of neonate CSMFs is illustrated in Figure 14 CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 27 Figure 14 Cause-specific mortality fractions for neonates (0–28 days)

Cause-specific mortality fractions for children (29 days to 11 years) In total, 239 VAs were conducted for children aged 29 days to 11 years. The main CODs were:

■ Drowning (26 per cent) ■ Pneumonia (16 per cent) ■ Cardiovascular disease (12 per cent) ■ Other defined cause of child death (10 per cent) ■ Cancer (8 per cent) ■ Other infectious disease (5 per cent) ■ Diarrhoea/dysentery (5 per cent) ■ Digestive disease (4 per cent) ■ Road traffic accident (3 per cent) ■ Meningitis (2 per cent) ■ Falls (1 per cent).

The distribution of childhood CSMFs is illustrated in Figure 15. CRVS Fellowship reports and profiles Fellowship CRVS

28 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Figure 15 Cause-specific mortality fractions, children (29 days to 11 years)

Leading causes of death for children (0–5 years) In total, 345 VAs were conducted for children aged 0–5 years. The main CODs were (Figure 16):

■ Pneumonia (28 per cent) ■ Undetermined (14 per cent) ■ Drowning (12 per cent) ■ Neonatal meningitis/sepsis (11 per cent) ■ Birth asphyxia (9 per cent) ■ Preterm delivery (7 per cent) ■ Congenital malformation (4 per cent) ■ Childhood cancer (4 per cent) ■ Digestive diseases (3 per cent) ■ Diarrhoea/dysentery (3 per cent). CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 29 Figure 16 Leading causes of death in children younger than 5 years CRVS Fellowship reports and profiles Fellowship CRVS

30 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Results: Plausibility of cause of death data generated through SmartVA

Comparison with Global Burden of Disease estimates

For most age groups, Global Burden of Disease (GBD) 2016 data show ratios of male to female deaths similar to those from SmartVA data. For men, GBD data show more deaths in the 0–28- day, 20–29-year, 29–30-year and >80 year age groups than do SmartVA date (Figure 17).

Figure 17 SmartVA data compared with Global Burden of Disease (GBD) estimates, adult males

For women, GBD 2016 data show more deaths in the0–28-dayand >80-year aged groups than do SmartVA data (Figure 18).

Figure 18 SmartVA data compared with Global Burden of Disease (GBD) estimates, adult females CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 31 The leading causes of death (CODs) in the SmartVA data were similar to those found in GBD2016 data.12 SmartVA results found more deaths due to ischaemic heart disease, stroke, chronic respiratory disease and diabetes than GBD data (Figure 19).

Figure 19 Comparison of leading causes of adult deaths, SmartVA data and GBD estimates

Table 7 Comparison of leading causes of adult death by sex, VA data and GBD estimates

COD = cause of death; GBD = Global Burden of Disease; NCD = noncommunicable diseased

12 GBD data redistribute ‘undetermined’ CODs. Therefore, SmartVA data were also redistributed to remove all undetermined

CRVS Fellowship reports and profiles Fellowship CRVS causes (18 per cent of all deaths).

32 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Comparison with Health and Demographic Surveillance System data

The data from SmartVA and the Health and Demographic Surveillance System (HDSS) were compared. Some differences in COD distribution were apparent (Figure 20, Table 8). Ischaemic heart disease was the leading COD in SmartVA, whereas stroke was the leading COD in HDSS. The top 20 COD rankings in HDSS are different from those found using SmartVA, except for road traffic accident (Annex 2, Table 12). SmartVA data provide more detail about different types of cancer; HDSS data group cancers together under ‘neoplasms’, which was ranked as the third-highest leading COD (10 per cent).

Figure 20 Comparison of leading causes of adult deaths, SmartVA and HDSS data

HDSS = Health and Demographic Surveillance System; NCD = noncommunicable disease

Table 8 Comparison of leading causes of adult death by sex, SmartVA and HDSS data CRVS Fellowship reports and profiles

COD = cause of death; COPD = chronic obstructive pulmonary disease; HDSS = Health and Demographic Surveillance System

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 33 Undetermined cases

As VA data is less The number of undetermined cases was 1437 (18 per cent) – 1372 in adults (18 per cent), 48 in specific than what children (20 per cent) and 17 in neonates (10 per cent).The number of undetermined CODs was is provided through highest in adults >80 years of age (Figure 21). The absolute number of undetermined CODs was medical certification, higher for men than women in nearly all age groups. some deaths cannot have a cause assigned. Figure 21 The distributions of undetermined causes of death, by age group, for each sex

Figure 22 shows the distribution of undetermined deaths in children aged 0–4 years – over one- third occur in the first 28 days.

Figure 22 Undetermined causes of death, by age group and sex, children <5 years old CRVS Fellowship reports and profiles Fellowship CRVS

34 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Results: Qualitative study

Five verbal autopsy (VA) supervisors and 16 VA interviewers were interviewed for the qualitative study.

Verbal autopsy supervisors

Observation of verbal autopsy interviews All supervisors claimed to observe every fifth VA. Their monthly transport claim history supported this.

Some of the main issues during VA were:

■ A lack of warmth when the interviewer met the respondent ■ The interviewer failing to sit face to face with the respondent ■ Respondents failing to provide answers while they are working, which also interrupts the VA

■ Interviewers who were too emotional to continue the normal flow of the VA interview ■ Interviewers who could not type smoothly on the tablet, which meant they were preoccupied with the tablet while conducting the interview.

Sometimes, it took a long time for the interviewers to receive the VA identification (ID), because the local government office was slow in responding. An assistant health inspector stated that it was therefore important to have regular coordination meetings with the local government office.

Two of the supervisors mentioned that the decedent’s relatives did not want to collect the death certificate from the hospital or local government office, as they were not aware of the importance of death certificates. This impeded the VA process.

Use checklist for supervision Very few supervisors Most (4 out of 5) supervisors did not use the supervisor checklist because they forgot to bring it used the checklist with them. for supervision. Discarded verbal autopsy data The VA supervisors did not discard any data, but VA was not conducted in a few instances (1–3 cases per supervisor). Reasons included hospital deaths (for which a VA was not required), double VA identifications (IDs) and absent cases. A health inspector explained why VA IDs were being doubled up:

Sometimes the deceased person’s relative goes to the local government registration (LGR) CRVS Fellowship reports and profiles office to drop off the death registration or death certificate. Another time, the health field worker goes to the LGR office to take the death registration of the same person. There is no way to avoid duel registration of the same person in the data entry system.

Another supervisor explained that sometimes respondents moved around during their relative’s death and subsequent burial, which made them difficult to locate when the VA was due.

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 35 Common tablet errors Some interviewers could not correctly spell the names of some diseases. This was managed by Technical issues with the tablets was a using a mobile phone to ask the assigned doctor the spelling. common challenge. Another problem was that the interviewer has to enter the family address twice.

There were also technical issues, including faulty tablets, changed URLs, exhausted data balances and the Open Data Kit software missing from the tablet. Problems were resolved by contacting the IT people of the upazila health complex.

Other verbal autopsy issues Another issue was that there was no proof that the VA interview actually took place.

One interviewer commented that understand the question was difficult to understand, such as the respondent's relationship to deceased. For example, If the deceased was the ‘father’ and the respondent was the ‘son’, they did not know whether to select father or son.

Debriefing session Most (3 out of 5) supervisors organised 2–7 debriefing sessions. During these sessions, they addressed field challenges, such as how to deal with the unwillingness of respondents to answer some questions. Some respondents wanted to know about the financial benefit of doing VA. When the interviewer explained the purpose, respondents agreed to help VA activities.

Sometimes it was difficult for the respondents to answer some of the questions, especially about diseases and disease names. Even if the decedent went to the doctor or hospital, the respondent may not have been told the diagnosis.

One supervisor mentioned that poor people may be upset with the hospital services, but are concerned that any negative feedback would affect them receiving services in the future. Therefore, they would refuse to answer questions about hospital services.

Monthly meetings All VA supervisors said they regularly discussed how to correctly complete VAs at the monthly meetings. This included identifying the best respondent, and the importance of answering all the questions.

All VA supervisors agreed that the proportion of death registrations has improved and cause of death (COD) data are now more readily available:

We can get the area-based mortality data of all the people at a glance.

The supervisors mentioned some challenges such as locating respondents when the family has It was often difficult to locate the family migrated. For example, if someone dies, the family may choose to return to the village to bury to organise an the body. Once the rituals are completed, the family leaves the village. This makes it difficult to interview time. meet with the best respondent. CRVS Fellowship reports and profiles Fellowship CRVS

36 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Verbal autopsy interviewers

Identifying death events Most of the VA interviewers received information about death events during house-to-house visits or when they had come to a community to make information announcements. Some were informed about deaths during vaccination sessions, by talking with the people, and from the community clinic.

Setting up the verbal autopsy interview All the interviewers said they set up the date and time of interview after talking with the deceased’s family.

When the interviewer received the information about a death, they visited the deceased person’s house to fill out the death registration form. They submitted the form to a union supervisor, who then submitted it to the LGR office. The LGR office uploaded the form and supplied a 17-digit VA ID number to the supervisor, who passed it to the VA interviewer. Next, the interviewer went to the deceased person’s house to set the date and time of the interview. Finally, the interviewer and supervisor went to the deceased person’s house to do the VA interview on the scheduled date.

Verbal autopsy refusals

Very few people All the interviewers agreed that there were only a few cases in which VA was refused, or VA refused to do a could not be done because the family members were absent or had migrated to other areas. VA interview. Travelling to places of interviews Most (14 of 16) interviewers said VA was not part of their normal work schedule. Most health assistants worked:

■ 3 days per week to collect health-related information by going house to house ■ 2 days per week at the vaccination centre ■ 1–1.5 days per week at the community clinic.

This schedule meant that they visited each household in a given area once per month, and their duties included collecting birth and death notifications. Often, VA could not be completed within the recommended timeframe because they needed to set aside at least 2 days to complete the process.

Two VA interviewers felt that they should fit the VA in regardless, as the VA program was new and part of the Directorate General of Health Services.

Details of the verbal autopsy interview All the VA interviewers said the best respondent was the person who could best answer the questions and was present at the time of death. CRVS Fellowship reports and profiles

Most interviews took place at the house of the deceased person. On average, an interview lasted 40–60 minutes, but this estimate may or may not have included travel time, identifying the best respondent (if not already identified), building a rapport with the respondent, obtaining consent and so on. Analysis of 500 random VAs suggested the average time to complete the interview was 25 minutes.

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 37 Difficulties during the interview process Most (12 of 16) VA interviewers said they face some difficulties while asking about the signs and symptoms of the deceased before death:

Some respondents do not want to answer questions about some causes of death, such as AIDS or suicide. After confirming confidentiality, [the respondent] agreed to answer.

There was some Some mothers did not want to answer questions without permission from her son. Some reluctance from family husbands do not want to say the exact reason for his wife's death due to fear. Sometimes members to talk about there was a problem collecting information about suicides. In these cases, respondents did certain deaths. not tell the truth. I talk to other people in the house and I try to find out the actual cause with their help.

Sometimes the respondent was too emotional to continue the interview.

Two interviewers noticed that respondents did not want to answer some questions about female deaths. This could have been because the interviewer was male.

Other times, the respondent did not know the answer because the deceased person had died suddenly.

Half the interviewers reported that some respondents could not answer many of the VA questions because the decedent was elderly and had never been ill, gone to a doctor or informed them of their health concerns.

Tablet errors All VA interviewers recorded the answers directly on the tablet. Some of the interviewers noted that there was no way to enter some of the CODs or symptoms, such as high blood pressure, type of cancer, fever for neonates and jaundice in the absence of yellow eyes, as these types of signs and symptoms questions don’t exist in SmartVA.

Link between death notifications and registrations, and verbal autopsy Most (15 of 16) interviewers said all VA cases were connected to a death notification/ registration. Some factors inhibited receiving a registration number in time for the VA:

■ VA registration numbers were often not available in time because of problems with servers, electricity supply or the local administration.

■ If the notifications are submitted at the end of the month (eg January), and VA number is to be collected during the month after (eg March), then the registration time is more than 45 days. CRVS Fellowship reports and profiles Fellowship CRVS

38 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Reporting and managing field challenges Most (13 of 16) interviewers discussed field challenges with their supervisors.

Some (7 of 16) interviewers mentioned that they had to visit a deceased person’s house several times, because information, or documents such as the National Identity Card and National Birth Registration Card, could not be found. Sometimes the best respondent could not be found. Some respondents wanted to know whether they would receive any benefit for their participation in VA.

Another VA interviewer suggested that people need to be made aware of the program – some did not want to help because the program is new and unfamiliar.

Most of the interviewers tried to solve problems by discussing them with their supervisors and the Upazila Health & Family Planning Officer. Sometimes the VA interviewer talked to other people in the house to help determine the COD. Interviewers also explained the aims of the VA to the community, to increase understanding of VA.

Improvements in the verbal autopsy process During discussions with their supervisors, most interviewers revealed that:

■ Death registrations were timely and correct ■ Knowledge about the most common CODs in an area is increasing ■ Interest in death registration is increasing ■ Recognition that the VA process helps the authorities plan for the future was increasing. CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 39 Discussion

Using SmartVA in Bangladesh has resulted in improvements in death registration and identifying causes of death (CODs). For example, in Kaliganj Upazila, 84 per cent of registered deaths were followed by verbal autopsy (VA).

The study found that noncommunicable disease (NCD) was a more common COD in adults than were communicable disease and injury-related deaths. NCDs may be caused by lifestyle changes, diet, environmental pollution and increased stress levels. It has been estimated that 59 per cent of deaths in Bangladesh (886 000 deaths) each year are caused by NCDs.13

The study revealed that the three most common CODs (ischaemic heart disease, stroke and chronic respiratory disease) are the same as those identified in the Global Burden of Disease study. However, the leading CODs in Health and Demographic Surveillance System (HDSS) data do not match those from the SmartVA data. Stroke was the leading COD in HDSS, followed by ischaemic heart disease and neoplasm. An analysis of 4969 VAs from Matlab HDSS (2011–14) found that stroke was the COD in 29.6 per cent of adult cases and ischaemic heart disease in 14.2 per cent of adult cases.14 Possible reasons for the differences between SmartVA findings and other data sources include:

■ DDifferences between target populations in the comparison groups, including different lifestyles and geographical characteristics

■ Differences in disease trends over time ■ Differences in data sources and analytical methods ■ Variability in the quality of VA interviews and the experiences of the interviewers ■ Difficulties in distinguishing diseases – for example, stroke is often mistaken for ischaemic heart disease.

The sex-specific, cause-specific mortality fraction SmartVA data showed some similarities with other studies. In the SmartVA study, ischaemic heart disease was the most common COD for men and stroke was most common for women. Another study found that stroke was more common among women and cardiac disease was more common among men.15 SmartVA shows that deaths from road traffic accidents and lung cancer are more prominent in men than in women, while the converse is true for diabetes. Another study also found that deaths from respiratory system neoplasms and road traffic accidents were higher among males, and from diabetes mellitus significantly higher among women.16

The SmartVA data revealed few maternal deaths. This could be because Bangladesh has significantly reduced maternal mortality and improved maternal mortality surveillance (including subjecting most maternal deaths to a separate maternal mortality VA process).

The SmartVA study did not include enough children and neonates to allow comparison with other studies, because they usually die from acute emergency conditions and these deaths are often reported in the hospital. Even so, SmartVA findings have some similarities to other studies. The SmartVA study found drowning to be the leading COD in children. In Bangladesh, drowning is the leading COD in children between 1 and 4 years old (43 per cent of deaths).17

13 Hazard R et al. The quality of medical death certification of COD in hospitals in rural Bangladesh: impact of introducing the International Form of Medical Certificate of COD. BMC Health Services Research. 2017; 17:688. 14 icddr,b. Non-communicable diseases. Available at: www.icddrb.org/news-and-events/press-corner/media-resources/non- communicable-diseases (accessed 1 February 2019). 15 Alam N et al. CODs in two rural demographic surveillance sites in Bangladesh, 2004–2010: automated coding of verbal autopsies using InterVA-4. Global Health Action. 2014; 7:25511. 16 Alam N et al. Distribution of COD in rural Bangladesh during 2003-2010: evidence from two rural areas within Matlab Health and Demographic Surveillance site. Global Health Action. 2014; 7:25510.

CRVS Fellowship reports and profiles Fellowship CRVS 17 Merelli A. Bangladesh has a plan to stop 18,000 kids from drowning every year. 15 April 2015. Quartz India.

40 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Another study found accidental drowning to be the COD in 49 per cent of childhood deaths aged 1–4 years.18 Pneumonia was the leading COD in neonates in SmartVA, followed by meningitis/sepsis, birth asphyxia, preterm delivery and congenital malformation. Another study found neonatal deaths were mostly due to acute respiratory infection/pneumonia, followed by birth asphyxia, prematurity and sepsis.19 These differences could be due to the computer-automated method (SmartVA-Analyze) used to analyse the SmartVA data.

The reasons for overrepresentation of men in VAs need to be explored, to find out why female deaths are not reported as often as male deaths.

An undetermined COD was assigned in 18 per cent of the cases, mostly in those older than 60 years. Common reasons included insufficient information, quality of the VA interview, lack of understanding of medical condition, no history of seeking health care, multiple comorbidities and confidentiality issues. In another study, Ifakara HDSS identified 5027 deaths from 2002 to 2007in Tanzania.20 The study found 31 per cent of CODs were undetermined when using physician-certified VA, mostly in children and adults. Factors associated with undetermined CODs were age at death, the level of education of the respondent, relationship with the respondent and place of death. The difference in percentage of undetermined CODs suggests that automated VA is capturing community CODs better than physician-certified VA.

The qualitative study identified some challenges, such as duplication of death registrations because no automated system to detect duplicate sexists. The only question the respondents found difficult to understand was related to ‘respondent's relationship to deceased’.

In SmartVA, the interviewers were asked to use a local term (if applicable) if the respondent did not understand any term, and to probe further to help them understand the question. There were some difficulties asking about HIV/AIDs, which another study also identified as a problem, because of the stigma associated with such conditions.21

Sometimes, the best respondent was not available to do the interview on the scheduled day, and the VA interviewer had to speak to someone else.

Although the SmartVA project has been scaled up in Bangladesh, VA is not yet included in the job descriptions of health workers; however, identifying birth and death registrations is a part of their regular work. Lack of support from the local registration office in getting a VA ID delayed the verbal autopsy. No system exists for reporting on supervision and monitoring in the field. The current tool does not have any automated monitoring system, just a VA supervisor checklist. Incentive is another issue that could affect staff motivation, the quality of the VA interview and the authenticity of the interviewer.

There are no dedicated, local IT staff to solve technical issues, such as problems with the server. Most of the trained IT people are working at the central level and have other issues to address,

making it hard to resolve SmartVA IT issues at the regional level in a timely manner. CRVS Fellowship reports and profiles

18 Alam N et al. CODs in two rural demographic surveillance sites in Bangladesh, 2004–2010: automated coding of verbal autopsies using InterVA-4. Global Health Action. 2014; 7:25511. 19 ibid. 20 Mwanyangala M et al. Verbal autopsy completion rate and factors associated with undetermined COD in a rural resource-poor setting of Tanzania. Population Health Metrics. 2011; 9:41. 21 llotey P et al. Let’s talk about death: data collection for verbal autopsies in a demographic and health surveillance. Global Health Action. 2015; 8:28219.

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 41 Common problems affecting verbal autopsy in Bangladesh

We grouped the common problems that affect VA in Bangladesh into themes (Figure 23).

Figure 23: Common themes affecting verbal autopsy in Bangladesh CRVS Fellowship reports and profiles Fellowship CRVS

42 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Recommendations to improve SmartVA in Bangladesh

Improving VA is a continuous process. Regular coordination and support from health departments, local registry officials and government representatives is crucial to increasing SmartVA implementation in Bangladesh.

Recommendations to improve SmartVA include:

■ A huge number of deaths occur at home, so it would be cost-effective if the government applied VA on a nationally representative sample of deaths to generate population-level COD data.

■ Errors could be minimised by establishing an automated system to flag any duplication of death registration and generate a unique VA ID for each case.

■ Activities related to VA should be included in community health workers’ job descriptions to incorporate VA in mortality surveillance routinely.

■ On-the-job and refresher training for VA interviewers and supervisors is necessary to maintain the quality of the interviews and thus improve the quality of VAs.

■ A proper monitoring and evaluation framework is important for appropriate implementation of VA to generate high-quality COD data.

■ Technical support services such as IT support should be geared up (both at central and regional level) to quickly resolve any issues related to VA.

■ Improving public awareness about the importance of COD and the role of VA would facilitate the nationwide scale up of VA in Bangladesh.

The supervisors provided some more specific suggestions to improve VA:

■ Include the ability to enter options such as son, daughter, father-in-law, mother-in-law and daughter-in-law for the ‘Relationship to the deceased’ question on the tablet.

■ Include dropdown options to enter the name of the disease in the tablet. ■ Include the ability to take a picture of the respondent or the home for the authentication of the interview.

Study limitations

It was not possible to determine the completion rate of VAs for all upazilas, because we did not have access to the total registered deaths, which only the local civil registry could obtain. The average time taken to conduct VAs could not be estimated because of technical issues with how the data are saved and downloaded from the server.

About 100 VA IDs could not be analysed at the upazila level because of a geographical coding CRVS Fellowship reports and profiles error. Four VA IDs were discarded from the analysis because the age data were invalid.

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 43 Annex 1 SmartVA implementation areas CRVS Fellowship reports and profiles Fellowship CRVS

44 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 10 Falls (N/%) (6/1.6) (5/1.6) (3/1.0) (6/2.2) cancer cancer (17/1.3) (16/1.8) (18/1.4) (22/1.5) (24/1.9) Cirrhosis Leukaemia Other NCD Pneumonia Renal failure Lung cancer Oesophageal Oesophageal 9 Other (N/%) (7/1.9) (5/1.6) (4/1.3) (6/2.2) cancer cancer (20/1.5) (17/1.9) (21/1.7) (22/1.5) (31/2.4) cancers Prostate Cirrhosis dysentery Diarrhoea/ Other NCD Pneumonia Road traffic Lung cancer Oesophageal 8 TB Falls Falls Renal (N/%) failure (8/2.1) (6/1.9) (4/1.3) (7/2.6) (33/2.5) (17/1.9) (23/1.8) (26/1.8) (36/2.8) Cirrhosis Maternal Pneumonia Road traffic Road Road traffic Road 7 Falls Falls NCD NCD NCD Lung Renal Other Other Renal Other (N/%) failure failure (6/1.9) (5/1.7) (7/2.6) cancer (37/2.8) (19/2.2) (29/2.3) (27/1.9) (10/2.7) (38/3.0) Cirrhosis 4No other data available 6 Falls Falls (N/%) (8/2.5) (9/3.1) (7/2.6) cancer cancer (37/2.8) (22/2.5) (31/2.5) (38/2.7) (10/2.7) (41/3.2) Prostate Cirrhosis Cirrhosis Cirrhosis Cirrhosis Other NCD Oesophageal 5 Falls Road Road Renal Renal Renal traffic traffic (N/%) failure failure failure (38/2.9) (28/3.2) (36/2.9) (61/4.4) (13/3.5) (47/3.7) (13/4.1) (11/3.8) (12/4.5) Diabetes Diabetes Diabetes Top 10 causes of death Top 4 Renal (N/%) failure (44/3.3) (43/5.0) (52/4.2) (70/5.0) (23/6.2) (61/4.8) (16/5.1) (12/4.1) (16/6.0) Diabetes Diabetes Diabetes Diabetes Diabetes Diabetes Cirrhosis Pneumonia 3 IHD (N/%) Stroke Stroke Stroke (26/9.0) (24/9.0) (1/14.2) Chronic Chronic Chronic Chronic Chronic (41/11.1) (35/11.2) Drowning (226/17.4) (106/12.3) (189/15.3) (192/13.8) (166/13.2) respiratory respiratory respiratory respiratory respiratory 2 IHD IHD (N/%) Stroke Stroke Stroke Stroke Stroke (2/28.5) Chronic Chronic Chronic (53/14.4) (40/12.8) (63/21.9) (38/14.3) (268/20.6) (136/15.8) (189/15.3) (244/17.6) (196/15.6) respiratory respiratory respiratory 1 IHD IHD IHD IHD IHD IHD IHD (N/%) Stroke Stroke Stroke (2/28.5) Chronic (08/23.7) (74/20.1) (81/25.9) (90/31.3) (49/18.5) (174/20.3) (227/18.4) (295/21.2) (203/16.2) respiratory 7 CRVS Fellowship reports and profiles 876 382 323 328 309 1387 1287 1423 1416 VAs VAs conducted date VA start VA Jan-17 Oct-17 Oct-17 Oct-17 Sep-17 Oct-17 Dec-17 Dec-17 Dec-17 Jan-18 Kaliganj Kaliakoir Kapasia Sreepur Upazila Sadar, Sadar, Gazipur Trishal Bhaluka Bishwanath Phultala Kishoreganj IHD = ischaemic heart disease; NCD = noncommunicable disease; TB = tuberculosis; VA = verbal autopsy = verbal IHD = ischaemic heart disease; NCD noncommunicable TB tuberculosis; VA Table 9 Leading causes of death, by upazila 9 Leading causes of death, by Table Annex 2 Statistical tables Annex

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 45 Table 10 Undetermined causes of death (CODs), by upazila

Undetermined COD Upazila Total number of VAs N % Kaliganj 1387 139 10

Kaliakoir 876 195 22 Kapasia 1287 314 24 Sreepur 1423 228 16 Sadar, Gazipur 382 83 22 Trishal 1416 260 18 Bhaluka 323 68 21 Bishwanath 328 74 22 Phultala 309 60 19 Kishoreganj 7 0 0

VA = verbal autopsy

Table 11 Leading causes of adult deaths, SmartVA data and Global Burden of Disease (GBD) estimates

SmartVA (2017–18) GBD (2016)

Male Female Both Male Female Both Cause of death (%, N = 4 (%, N = 2 (% N = 7 Cause of death (%, N = (%, N = (%, N = 691) 733) 425) 438 918) 299 938) 738 855) Ischaemic heart disease 24 18 22 Ischaemic heart disease 18 17 17 Stroke 17 22 19 Stroke 16 19 17 Chronic respiratory 17 15 16 Chronic respiratory 12 11 12 Diabetes 5 6 5 Other noncommunicable 7 8 8 disease Other noncommunicable 3 4 4 Other cancer 5 5 5 disease Renal failure 4 3 4 Diabetes 3 6 4 Cirrhosis 3 3 3 Cirrhosis 5 2 4 Road traffic 4 2 3 Diarrhoea/dysentery 3 5 3 Falls 2 3 2 Other infectious disease 3 4 3 Pneumonia 2 2 2 Lung cancer 4 2 3 Lung cancer 2 1 2 Road traffic accident 5 1 3 Other infectious disease 1 2 1 Other cardiovascular 3 4 3 disease Tuberculosis 2 1 1 Tuberculosis 4 1 3 Oesophageal cancer 1 1 1 Renal failure 2 3 3 Other cancer 1 2 1 Pneumonia 2 2 2 Prostate cancer 2 0 1 Suicide 1 2 2 Suicide 1 2 1 Colorectal cancer 1 1 1 Drowning 1 1 1 Drowning 1 1 1 Leukaemia/lymphoma 1 2 1 Other injury 1 1 1 Maternal 0 2 1 Stomach cancer 1 1 1 CRVS Fellowship reports and profiles Fellowship CRVS

46 Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 Table 12 Leading causes of adult deaths, SmartVA and HDSS data

SmartVA (2017–18) HDSS (2017)

Male Female Both Male Female Both Cause of death (%, N = (%, N = (%, N = Cause of death (%, N = (%, N = (%, N = 4691) 2733) 7424) 736) 616) 1352) Ischaemic heart disease 22 15 19 Stroke 25 33 29 Undetermined 17 21 18 Ischaemic heart disease 20 14 17 Stroke 15 19 16 Neoplasm 14 6 10 Chronic respiratory 16 13 15 Other cardiovascular 7 10 8 Diabetes 4 5 4 Unknown/missing 5 9 7 Renal failure 3 3 3 COPD 7 3 5 Cirrhosis 3 2 3 Septicaemia 2 5 3 Falls 2 3 2 Digestive disease 3 3 3 Road traffic 3 1 2 Unintentional injury – 2 3 2 accident Other noncommunicable 2 2 2 Diabetes 2 3 2 disease Pneumonia 2 2 1 Tuberculosis 2 1 1 Lung cancer 2 1 1 Hepatitis 2 1 1 Oesophageal cancer 1 1 1 Diarrhoeal 1 1 1 Prostate cancer 2 0 1 Intentional injury – 2 1 1 suicide Leukaemia/lymphoma 1 1 1 Renal failure 1 1 1 Tuberculosis 1 1 1 Neoplasm in female 0 2 1 organ Maternal 0 2 1 Other respiratory 1 1 1 Other cancer 1 1 1 Unintentional injury – 1 1 1 drowning Other injuries 1 0 1 Respiratory infection 1 1 1 Breast cancer 0 2 1 Intentional injury – 1 1 1 homicide

COPD = chronic obstructive pulmonary disease; HDSS = Health and Demographic Surveillance System CRVS Fellowship reports and profiles

Fellowship report: Integrating verbal autopsy in routine mortality surveillance in Bangladesh | Version 1319-01 47 48 CRVS Fellowship reports and profiles Topic 4:CauseofdeathinCRVS –Automated verbal autopsy. https://crvsgateway.info/learningcentre Learning Centre University ofMelbourne,D4HInitiative, CRVS Knowledge Gateway: SmartVA. Medical certificationofcausedeath. https://crvsgateway.info/courses University ofMelbourne,D4HInitiative, CRVS Knowledge Gateway: Courses Fellowship report: Integratingverbal autopsy inroutine mortalitysurveillance inBangladesh |Version 1319-01 Introducing verbal autopsiesintoCRVS:guidingprinciples.CRVS technical outcome series. Intervention: Medicalcertificationofcausedeath.CRVS summaries. Intervention: Improving registration practices. CRVS summaries. Intervention: Automated verbalautopsy. CRVS summaries. CRVS countryoverview:Bangladesh. summaries. Challenges associatedwithautomatedVA training androllout. CRVS development series. Bangladesh: AsuccessfuljournaltowardsCRVSsystemimprovement. CRVS development series. https://crvsgateway.info/library University ofMelbourne,D4HInitiative, CRVSKnowledge Gateway: Library Related resources andproducts The program partners on this initiative include: The University of Melbourne, Australia; CDC Foundation, USA; Vital Strategies, USA; Johns Hopkins Bloomberg School of Public Health, USA; World Health Organization, Switzerland.

Civil Registration and Vital Statistics partners:

For more information contact: [email protected] crvsgateway.info

CRICOS Provider Code: 00116K

Version: 0319-01

Copyright © Copyright University of Melbourne March 2019. The University of Melbourne owns the copyright in this publication, and no part of it may be reproduced without their permission.

Disclaimer The University of Melbourne has used its best endeavours to ensure that the material contained in this publication was correct at the time of printing. The University gives no warranty and accepts no responsibility for the accuracy or completeness of information and the University reserves the right to make changes without notice at any time in its absolute discretion.

Intellectual property For further information refer to: unimelb.edu.au/governance/statutes

unimelb.edu.au