MEASURE Evaluation September 2019

Building a Strong and Interoperable Digital Information System for

Background Summary “Interoperability” denotes the ability of two or more infor- Uganda launched its National eHealth Policy mation systems to exchange information and work together and Strategy in May 2018.1 Leadership and in and across organizational boundaries to advance the health governance, workforce development, enterprise status of individuals and communities. Interoperability also architecture, health information systems integration, underpins effective delivery of healthcare, by enabling clients and interoperability are the strategy’s key areas of to move across service delivery points in the same health facil- implementation. Most important, the strategy calls ity or move to other health facilities and access their records for “harmonized eHealth initiatives at all levels,” (Healthcare Information and Management Systems Society, given the various digital health information systems 2013). (HIS) that have taken root in the country. To build a strong HIS reinforced by digital health, the MOH To understand how to prioritize investments and of Uganda has developed an eHealth strategy and framework, implementation toward interoperability within these and has established a technical working group (TWG) that systems, the Ministry of Health (MOH) of Uganda, meets regularly to guide this work. Within the eHealth with support from MEASURE Evaluation—funded framework, certain challenges impede rapid implementation by the United States Agency for International of eHealth: fragmented data sources, numerous and Development (USAID)—conducted a readiness uncoordinated digital health pilot projects, poor computer assessment of the interoperability of Uganda’s HIS literacy skills, data security and privacy concerns, a shortage with in-country HIS stakeholders. of health staff, and systems that are unable to exchange data. The interoperability readiness assessment is one step toward The assessment team focused on three major determining the status of the HIS and its domains, and domains of an HIS: leadership and governance, the assessment results contribute to the development of a human resources, and technology. They used roadmap to strengthen the weak areas. (See Figure 1.) the Health Information System Interoperability Maturity Toolkit,2 developed in 2017 by MEASURE Figure 1. The domains and components of an Evaluation and the Collaborative’s interoperable HIS Digital Health and Interoperability Working Group, with input from digital health stakeholders in Ghana and Kenya. A maturity model measures the ability of an organization or government entity, such as a health ministry, to continuously improve in a specific discipline until it reaches the desired level of development, or maturity. Using the results of the assessment, the team brainstormed activities that Uganda could prioritize to move the country toward a stronger, interoperable digital HIS. This brief describes the assessment process, results, and recommended actions.

1 Available at http://www.health.go.ug/content/national- MEASURE Evaluation used the maturity model approach ehealth-strategy to conduct the assessment and analyze the results. The 2 Available at https://www.measureevaluation.org/resources/ maturity model comprises a set of structured levels that depict tools/health-information-systems-interoperability-toolkit/health- information-systems-interoperability-toolkit organizational behaviors, practices, and processes that reliably and sustainably produce required outcomes (Hammond, Bailey, Boucher, Spohr, & Whitekar, 2010). MEASURE Evaluation September 2019

Assessment revealed that most subdomains are at the lowest two levels: nascent or emerging. At the nascent level, HIS activities MEASURE Evaluation facilitated the HIS interoperability happen by chance or represent isolated and ad hoc efforts. readiness assessment in Uganda in June 2018. The assessment The emerging level characterizes a system with defined HIS was carried out with the two-step process described below. processes and structures. However, such processes are not 1. Formation of an assessment oversight team to plan systematically documented and lack ongoing monitoring or the assessment. The MOH worked with MEASURE measurement mechanisms. (See Table 3 for the detailed results.) Evaluation to identify health professionals with the right Several reasons explain the low scores in most of the digital expertise, experience, and authority to form the team. The health interoperability domains. specific mandate for the eight-member team (and two MEASURE Evaluation facilitators) was to provide the First, the scoring method: The overall domain score is equal overall scope and direction for the assessment, determine to the lowest score of its subdomains. For example, under which stakeholders would be invited for the assessment, and the leadership and governance domain, the subdomains for oversee the assessment. The MEASURE Evaluation team “governance structure for HIS” and “financial management” oriented the assessment oversight team to the assessment tools received a score of three and five, respectively. However, and processes. Table 1 lists the members of the assessment the subdomains for “HIS interoperability monitoring and oversight team. evaluation” and “compliance with data exchange standards” attained a score of one. Going by the logic that the overall 2. Assessment workshop. The assessment workshop was a domain score is equal to the lowest score of its subdomains, the one-day event attended by 27 participants (listed in Table 2), leadership and governance domain gets a net score of one. who represented the MOH and other government ministries, the United States Centers for Disease Control and Prevention Second, HIS scope: The wider the scope for HIS in the (CDC), the World Health Organization (WHO), and several assessment, the higher the likelihood of a lower score. Uganda of the MOH’s implementing partners. The MEASURE adopted a broad definition of HIS that includes not only the Evaluation team presented the assessment’s goals, scope, information systems for collecting patient-level information and and process. Then, each of the participants completed the human resources, laboratory, and logistics data, but also other assessment questionnaire. Next, the oversight assessment team auxiliary systems such as civil registration and vital statistics. facilitated a consensus-building session on the results. The With a mix of strong and weak systems, the weak systems pull aim of the consensus-building process was to develop a final down the net domain score. set of answers acceptable to all participants. The assessment Third, the extent of maturity level achievement: A results would then help determine the maturity levels of the subdomain gets a full maturity score only when the interoperability domains and their subdomains. requirements of the maturity level are fully achieved. Partial Results achievement does not count in the scoring. Partial achievement is, however, demonstrated by a plus sign (+) next to the The consensus-building process produced the final results maturity level score of a subdomain. In this assessment, there that were further analyzed into scores and mapped onto the were multiple subdomains with partially attained maturity maturity model, to produce a visual presentation of maturity levels (shaded in yellow in Table 3). levels. Each of the subdomains of the HIS interoperability maturity model has five levels of maturity: nascent, emerging, Figure 2 shows the subdomains and their respective levels, based established, institutionalized, and optimized. The results on the results of the assessment.

Figure 2. Summary of the results of the Uganda HIS interoperability assessment

Level 1: Nascent Level 2: Emerging Level 3: Level 4: Level 5: Established Institutionalized Optimized Compliance with data exchange standards Interoperability guidance documents Governance structure HIS interoperability monitoring and for HIS evaluation Data management Business continuity Communication Human resources policy Human resources network, local area capacity (skills and network, and wide Financial National HIS enterprise architecture numbers) area network management Technical standards Data ethics Hardware Financial HIS subsystems Operations and main- resource tenance for computer mobilization Human resource capacity development technology

2 MEASURE Evaluation September 2019

Solutions: The Three-Box “Framework for box. Group members contributed information for their box Innovation” Strategy on note paper, and then one group member read through the notes and grouped them by theme, forming affinity After the assessment results were agreed upon, the maps. Boxes 1 through 3 summarize each group’s notes. assessment oversight team held a workshop to discuss activities that Uganda could implement to move the country toward a stronger and interoperable digital HIS ecosystem. To do this, we used the “three-box framework” approach (see Figure 3). Box 1. How to optimize the present

Figure 3. The Three-Box Framework for Suggested activities for improving the current HIS: InnovationThree Box F rStrategyamework for Innovation Strategy • Identify and adopt drivers for interoperability of HIS. Three-box3-Box ba balancelance is req isui r requireded to mana toge managetension tension betweenbetween in ninnovationovation and e andxecut iexecutionon • Create a legal framework to support digital health interventions. Box 2 Box 3

Selectively • Develop standard operating procedures for HIS Create the Forget the Future and interoperability. Past • Retool current human resources approaches to BoxBo 2:x 2 :LetLe t gogo BoxBox 3 :3:B Builduild new of ovaluesf values aandnd Box 1 productsnew product sor or attune them to the needs of digital health. practicespractices t hthatat servicesservices to mtoee meett unnecessarilyunnecessarily Optimize emergingemerging nee dneedss • Provide digital health tools, such as computers and inhibitinhib itcreating creating the requiringrequiring no nnon-- theth efuture future linear solutions infrastructure. Present linear solutions • Implement Figure 4. Box 1: Optimizing BoxBox 11:: UUsese in incrementalcremental innova tinnovationsions to to the present

operateoperate be tbetter,ter, faster, faster,or cheap eorr cheaper 1 current guidelines for Source:Source: Govindarajan, Govindarajan, V. V. (2016). (2016). The The three-box solution: solution: A A strategy for for leading leading innovation. Cambridge, MA, USA: Harvard Business Press. digital health. innovation. Cambridge, MA, USA: Harvard Business Press. • Develop user-centered The principle of the three-box framework, designed by information Harvard University professor Vijay Govindarajan, is systems. used in finding ways to meet performance requirements for an existing business—one that is thriving—while • Provide dramatically reinventing it. The framework is a tool mentorship kit with a method of self-analysis for allocating an and supportive organization's energy, time, and resources—in balanced supervision. measure—across what Govindarajan calls the three • Build boxes. Box 1 represents the present: how to manage the mechanisms core business and maintain its peak performance. Box for strong 2 represents the past: how to carefully abandon ideas, governance practices, and attitudes that could inhibit innovation. and leadership. Box 3 represents the future: how to convert breakthrough • Develop systems that assure high-quality data. ideas into new products and businesses, and how to do things differently. This approach befits countries that are • Listen to and act on clients’ needs. implementing digital health solutions against a backdrop • Set up regular meetings with stakeholders to ensure of entrenched manual systems and practices. that their needs are met. • Train HIS officers. Approach • Provide HIS officers with the proper tools. To apply the three-box framework, stakeholders were divided into three groups—one for each of the three • Develop national guidelines for HIS. boxes. The facilitators gave each group a set of questions to • Define digital health standards. guide them in providing information for their respective • Mobilize financial resources.

3 MEASURE Evaluation September 2019

Box 2. How to selectively forget the past Box 3. How to create the future This box examines the practices that hold the This box is about building new products and services organization back from achieving better results. These to meet emerging needs. It’s about using innovative practices may have worked in the past but are no approaches to build a better future for the organization. longer relevant. In this framework, those practices are The following are ideas that support this process: referred to as “chains” that shackle the organization • Create interconnected information systems that are from advancing. It should be noted, however, that not results-oriented. all past practices are bad. Some practices that worked in the past are still viable for the continuous growth • Identify countries and programs that are good role models for digital health practices. of an organization, and Figure 5. Box 2: Current should be sustained. In impediments • Establish and use the open digital health standards. this framework, such • Gradually migrate from electronic medical records practices are referred to to personal health records. as “roots”: they provide an organization with • Train more HIS officers on digital HIS. needed support. In this • Update current digital HIS to adhere to best session, stakeholders practices. discussed practices that • Involve stakeholders Figure 6. Box 3: Ideas for could encumber the at every level. the Future development of a vibrant digital HIS. Below are • Establish the ideas generated by leadership and the group: governance mechanisms • Duplication of focused on activities current and future • Multiple information systems that serve the same information needs. purpose at health facilities • Design and • Overreliance on paper as the chief information implement system a nationally • Data and program silos approved HIS architecture. • Poorly coordinated efforts for strengthening HIS • Complete HIS • Low human resources capacity for digital health assessments in the country and use the results to • Lack of reliable infrastructure for digital technology rationalize current information systems and to • Low level of data use across all levels of health upgrade information systems. • Bias against electronic systems • Develop a clear plan to migrate from a paper-based system to a digital one. • Fear of change promised by digital heath • Use a phased approach to implement digital health • Information systems that are not interoperable systems. • Fragmented ownership of information systems • Create capacity and awareness of data use. • Implement ethical guidelines for digital data management. • Develop application programming interfaces to enable data exchange.

4 MEASURE Evaluation September 2019

The Road Ahead In addition, Uganda has embarked on e-Government projects and has some structures and artifacts in place, which the health Sustained and structured discussions on interoperability will sector can adapt or learn from. For example, the national guide implementation of activities that are critical to achieving information and communication technology policy outlines national interoperable HIS. Using the assessment results and business continuity and disaster recovery strategies for the content from the ensuing group discussions, the assessment country that the digital health ecosystem can use. oversight team started drafting the HIS interoperability roadmap and highlighted several areas for immediate work To sustain established momentum for eHealth, the Uganda under each of the domains assessed MOH should take the following steps: • Present the findings of this assessment to the Leadership and governance eHealth TWG for discussion and adoption. Seek the In the context of Uganda, it seemed logical for the commitment of the TWG to provide ongoing oversight implementation of digital health and interoperability activities of the development and implementation of the HIS to take place under the auspices of the eHealth TWG, interoperability roadmap, backed by a monitoring and backed up by a monitoring and evaluating framework. This evaluation framework. group provides the necessary digital health leadership and • Create a subcommittee focused on HIS interoperability, governance oversight. Secure funding for digital health and as a subset of the eHealth working group, to lead HIS interoperability work, supported by a costed, long-term interoperability work. work plan, will be critical to the success of interoperability. Additionally, once the adapted standards are published, • Complete the HIS interoperability framework/roadmap. compliance mechanisms need to be established and MEASURE Evaluation helped the oversight assessment enforced. Uganda has already launched a number of eHealth team to start drafting its responses, but more meetings and documents to shepherd the digital health and interoperability resources will be needed to complete this work and align it work—notably, an eHealth policy and eHealth framework. with other ongoing activities. In fact, the eHealth policy and strategy stipulate that specific • To implement the findings from the assessment, it is guidelines for interoperability standards should be developed imperative to develop a costed work plan and use it to for all players to follow. advocate resources. As a starting point, the Ministry of Health-Division of Health Information brought two Human resources volunteer health informatics specialists on board to support Without adequate numbers of competent digital health and the implementation of the eHealth policy and strategy HIS experts, it will be an uphill task to implement many of while other capacity building activities are ongoing. the activities in the interoperability roadmap, much less to Table 1. The oversight team sustain them. The MOH and its partners, and the eHealth Ministry/Agency/ TWG need to embark on long-term capacity-building Name Role Department activities for staff at all levels of the health system. A natural Ag. Commissioner starting point would be to develop or adapt normative Dr. Sarah Byakika Ministry of Health Health Services–Planning guiding documents for implementers to use. Department Ag. Assistant Technology Caroline Kyozira Ministry of Health Commissioner–Division of Health Information The assessment and follow-up discussions yielded a number Health Informatics Mpiima Jamiru Ministry of Health of items that need attention now. Data management, data Specialist exchange, and data security and privacy standards are urgently Expeditus Ahimbisibwe Ministry of Health Principal Health Planner needed to standardize digital health work across programs. Health Informatics Catherine Kabahuma Ministry of Health In addition, a long-term plan for and hardware Specialist maintenance is needed, as well as the continued involvement National Information Technology IT Application Solutions Emily Nakazi of National Information Technology Authority-Uganda Authority–Uganda Manager (NITA-U) to provide the data communication infrastructure. (NITA-U) Internationally acclaimed standards exist that can be adapted Lordwin Kasambula Ministry of Health M&E Specialist and published for the country’s use. In fact, NITA-U already Emmy Muramuzi Ministry of Health Senior M&E Specialist has reference standards for the ongoing work of establishing Sam Wambugu MEASURE Evaluation Facilitator e-Government projects; the health sector can adapt them. Christina Villella MEASURE Evaluation Facilitator

5 MEASURE Evaluation September 2019

Table 2. Assessment and roadmap workshop participants

Name Organization

Carol Kyozira MOH–Division of Health Information

Mpiima Jamiru MOH–Division of Health Information

Emily Nakkazi NITA-U

Catherine Kabahuma MOH–Division of Health Information

Lordwin Kasambula MOH–AIDS Control Program

Twesige Nowen Stephen NITA-U

Winfred Ingabire Ministry of Gender, Labour and Social Development

Juliet Tumwikirize USAID–Strategic Information Technical Support Project (SITES)

Akello Lilian Perry MOH–Human Resource

Akena Stephen Abwoye MOH–Division of Health Information

Dr. Henry Mwanje Uganda–UK Health Alliance

Expeditus Ahimbisibwe MOH–Planning department

Musenge Kenneth CDC

Moses Bagyendera WHO

Emmy Muramuzi MOH–AIDS Control Program

Vincent Ndizima CDC

Nic Luwgenjo USAID–SITES

Nsubuga John D.S. MOH

Kayanja Edward MOH–Division of Health Information

Lubwama Samuel CDC M&E Technical Support (METS)

Kyarisiima Dinnah MOH–Reproductive Health

Oyo Godfrey MOH–Human Resources

Muyingo Edmond MOH–Division of Health Information

Muwanguzi Samuel MOH–Information, Communications, and Technology (ICT)

Ssali Tamale Muzamiru MOH–ICT

Ebony Quinto MOH–National Tuberculosis and Leprosy Control

Lubowa Nathan MOH–Pharmacy Department

Irongo Daniel World Vision

Steve Wanyee IntelliSOFT Kenya

6 MEASURE Evaluation September 2019

Table 3. Uganda health information systems interoperability maturity assessment results

HEALTH INFORMATION SYSTEMS INTEROPERABILITY MATURITY MODEL WORKSHEET Domain Subdomain Level 1: Nascent Level 2: Emerging Level 3: Level 4: Level 5: Maturity The country lacks The country has defined Established Institutionalized Optimized Level HIS capacity or does HIS processes and The country has documented Government and stakeholders The government and not follow processes structures, but they are not HIS processes and structures. use the national HIS systems stakeholders routinely systematically. HIS systematically documented. The structures are functional. and follow standard review interoperability activities happen by No formal or ongoing Metrics for performance practices. activities and modify them chance or represent monitoring or measurement monitoring, quality to adapt to changing isolated, ad hoc protocol exists. improvement, and evaluation conditions. efforts. are systematically used.

Leadership Governance Evolving governing An HIS governing body The HIS governing The HIS governing body is The HIS governing body and structure for HIS body for health is formally constituted body conducts regular government-led, consults is legally protected Governance information systems and has a scope of work meetings with stakeholder with other ministries, and from interference or (HIS) is constituted on that includes the people participation. monitors implementation of organizational changes. a case-by-case basis responsible for data HIS interoperability using The HIS governing OR no governing governance oversight. a work plan. It mobilizes body and its TWGs are body exists. The governing body resources—financial, nationally recognized oversees interoperability human resources (HR), and as the lead for HIS 3+ directly or through a political—to accomplish its interoperability. The separate TWG. goals. governing body works in liaison with other similar working groups regionally and/or around the world. Interoperability HIS interoperability The governing body for Interoperability guidance The interoperability Processes are in place guidance guidance documents HIS interoperability has documents developed, guidance documents are to regularly monitor documents1 are absent, and HIS drafted the necessary HIS tested, and adopted, government-owned. They the implementation interoperability is interoperability guidance and include reference are consistently used and of the interoperability implemented on a documents. terminologies and referenced in efforts to guidance documents. The case-by-case basis. technical standards for guide implementation of interoperability guidance data exchange. HIS interoperability. documents are regularly reviewed and updated 2 based on lessons learned from implementation. These documents reflect international best practices.

1 The approved documents (policies, strategies, and frameworks) that guide HIS and digital health/eHealth work in a country.

Current subdomain level: The level at which all the attributes at that level and the levels below have been achieved

Level with all attributes achieved: Level above the current subdomain level with all attributes in that level achieved

Level with some attributes achieved: Level above current subdomain level with some attributes of that level achieved 7 MEASURE Evaluation September 2019

Level 4: Subdomain Domain Subdomain Level 1: Nascent Level 2: Emerging Level 3: Established Level 5: Optimized Institutionalized Level Leadership Compliance No structure, processes, and Structures (working The HIS has developed The government enforces Compliance with and with data procedures (e.g., working groups, steering or adopted and the regulatory framework standards for data Governance exchange groups, steering committees, committees, or units) implemented a for compliance. The exchange, messaging, standards or units) are in place to guide are in place to guide or regulatory framework for subsystems in the and security is regularly or enforce compliance with enforce compliance. compliance. national HIS are required reviewed. The regulatory data exchange, messaging, to meet compliance and framework is reviewed and data security standards. certification criteria. and updated to reflect 1 No criteria for certification best practices for data and compliance exist. No exchange, messaging, regulatory framework for and systems security. compliance exists.

Data ethics The country lacks HIS The country has defined The country has Government and The government and capacity or does not follow HIS processes and documented HIS stakeholders use the stakeholders routinely processes systematically. HIS structures, but they processes and structures. national HIS systems review interoperability activities happen by chance are not systematically The structures are and follow standard activities and modify or represent isolated, ad hoc documented. No formal functional. Metrics for practices. them to adapt to efforts. or ongoing monitoring performance monitoring, changing conditions. 2+ or measurement protocol quality improvement, exists. and evaluation are systematically used. HIS No tracking, or ad hoc The methods and HIS interoperability Mechanisms to track and Results from monitoring interoperability tracking, is done of HIS tools to report on activities are regularly measure performance of HIS interoperability M&E interoperability activities HIS interoperability monitored and of HIS interoperability are used for planning. related to plans, resources, implementation reviewed accordingly. work are government- and budgets for the national are defined and Regular reports on approved and Decisions about HIS. documented. HIS interoperability government-led. future activities take 1 performance are this analysis into generated and consideration. disseminated to stakeholders. Business No government-approved The HIS has developed The BCP has been The BCP has been The BCP has been continuity business continuity plan (BCP) a BCP that outlines audited. Audit results audited. Audit results audited. Audit results is in place at the national or the processes needed show that at least 50% show that at least 75% show that all or most subnational levels of the HIS. to ensure continuity of the BCP has been of the BCP has been of the BCP has been 2 of critical business implemented. implemented. implemented. processes.

8 MEASURE Evaluation September 2019

Level 4: Subdomain Domain Subdomain Level 1: Nascent Level 2: Emerging Level 3: Established Level 5: Optimized Institutionalized Level Leadership Financial No clear plan exists for High-level financial Detailed financial The HIS budget is part of An established, long- and management financial management management structures, management structures, the Ministry of Health’s term HIS financial Governance of HIS, including including budgets, including budgets for HIS budgeting process. management system interoperability activities. are developed for the interoperability at the is owned, reviewed, national HIS, including national andsubnational Financial audit processes tracked, and updated interoperability in the levels, are developed based are in place and are by the government, country based on HIS on the HIS work plan. HIS carried out regularly to and is supported by 5 work plans. expenditures are monitored promote accountability in stakeholders. against HIS budgets. HIS spending.

Financial There is no documented Financial resources A costed work plan at Government and Agovernment-owned, resource plan for financial for HIS strengthening, national and subnational implementing partners costed, long-term work mobilization resources for HIS including HIS levels is in place that covers have sufficient funding plan (five years or strengthening, including interoperability, are both the information and to implement the more) is in place to HIS interoperability. mostly donor driven. communications technology costed work plan. The support ICT and human (ICT) infrastructure (network, government owns the resources for HIS hardware, and software), costed work plan. strengthening, including and personnel for HIS HIS interoperability. A needed for HIS strengthening, mechanism is in place 3+ including HIS interoperability. to regularly review and At a minimum, this work update the work plan. plan identifies the activities, timeframe, costs, and sources of funding for HIS interoperability.

Domain maturity level: Leadership and Governance 1

9 MEASURE Evaluation September 2019

Subdomain Domain Subdomain Level 1: Nascent Level 2: Emerging Level 3: Established Level 4: Institutionalized Level 5: Optimized Level Human Human There is no HR policy A national needs An HR policy or strategic Implementation plans are in A long-term plan is in place Resources resources that recognizes assessment has been plan exists that identifies place for growing a cadre of to grow and sustain staff with policy HIS-related cadres. completed showing the the HIS, digital HIS, and staff at national and subnational the skills needed to sustain Distribution of HIS number of staff and interoperability skills levels for digital HIS and HIS and digital HIS and human resources is types of skills needed to and functions needed to interoperability. interoperability. Performance ad hoc. support HIS, including support the national HIS management systems are digital HIS and and its digital HIS and in place to monitor growth interoperability. interoperability. and sustainability of the HIS workforce. 1+ HIS-related cadre roles and responsibilities are mapped to the government’s workforceand schemes of work. Human The country has no The country depends The country has a The country has staff in sufficient The country has a sufficient resources dedicated cadre of on technical assistance growing staff with skills numbers with relevant skills and sustainable number of staff capacity staff for maintaining from external in governance and to support the digital HIS and with an appropriate mix of skill (skills and the digital HIS and stakeholders to support leadership, data collection, interoperability at national and sets to support the digital HIS numbers) interoperability. the national and data management, data subnational levels. and interoperability at national Responsibility for subnational digital HIS sources, health information and subnational levels, and the the HIS is added to and interoperability. technology (IT), and interoperability of key systems. existing positions. managing information A human resources for health 2+ products. The staff are strategic plan is in place to sufficient in numbers continuously upgrade staff and skills at the national skills to reflect international level, but inadequate at best practices in digital HIS subnational levels. and interoperability, preferably with locally generated funds. Human The country has no A nationally recognized A plan exists for in-service The country has the capacity Opportunities and incentives resource national training pre-service training training of HIS staff to to train enough staff to support are in place for continuing capacity programs to build curriculum exists build skills around digital digital HIS and interoperability, education in digital HIS and development human resource that outlines needed HIS and interoperability through in-country pre-service interoperability for HIS- related capacity on digital competencies for human based on a nationally or and in-service training institutions cadre staff, to keep them up-to- HIS, including resources for digital HIS internationally recognized or partnerships with other date as the HIS field evolves. interoperability. and the interoperability HIS curriculum. training institutions. Government 1+ of the HIS. and stakeholders provide sustainable resources for health ministry staff to receive training on HIS, including digital HIS and interoperability.

Domain maturity level: Human Resources 1+ 10 MEASURE Evaluation September 2019

Subdomain Domain Subdomain Level 1: Nascent Level 2: Emerging Level 3: Established Level 4: Institutionalized Level 5: Optimized Level Technology National HIS A national HIS A validated national HIS Foundational tools The government owns, The national HIS enterprise enterprise enterprise architecture enterprise architecture and rules for HIS enforces, and leads architecture and its ISL are architecture document defining exists that defines interoperability exist. implementation of the fully implemented using technology technology requirements national HIS enterprise industry standards. The requirements and data and exchange formats They include a health architecture, including the ISL provides core data exchange formats for for interoperability. It is information management ISL and core authoritative exchange functions and interoperability does validated, but not widely system for routine and registries (Facility Registry, is periodically reviewed not exist, or there is shared or systematically surveillance data, and Metadata Dictionary, and updated to meet the a draft document, applied by the HIS core authoritative registries Master Patient Index, and changing country data but it has not been community. (Facility Registry, Metadata Health Worker Registry). needs. There is continuous validated or shared Dictionary, Master learning, innovation, and 1+ with the country’s HIS Point to point data Patient Index, and Health quality control in the work community. exchange between some Worker Registry). The on HIS interoperability. HIS applications exists, Interoperability Service but there is no systematic Layer (ISL) for the HIS is implementation of the operational and provides agreed-upon architecture. core functions, such as data authentication, translation, an interpretation. Technical No defined technical An HIS ICT infrastructure An interoperability lab Technical standards for A routine review of standards² standards exist for use assessment has been exists for new partners to national data exchange standards and requirements in the country’s HIS conducted and the needs test technical standards have been published and compliance is conducted data exchange. for a coherent HIS ICT or for onboarding new disseminated in the country to ensure continuous infrastructure architecture HIS subsystems, and a under the government’s integration of the various Applications are have been documented. certification mechanism leadership. The ISL subsystems. hosted by the The country has adopted exists for new HIS is orchestrating data 1 providers without or developed technical subsystems to be integrated exchange between existing any control from the standards for health data in the national HIS. HIS applications hosted government or MOH. exchange, messaging, and by the integrated ICT security. infrastructure supporting the national HIS.

2 Including standards for data exchange, transmission, messaging, security, privacy, and hardware

11 MEASURE Evaluation September 2019

Subdomain Domain Subdomain Level 1: Nascent Level 2: Emerging Level 3: Established Level 4: Institutionalized Level 5: Optimized Level Technology Data No national document Electronic data A roadmap is in place to National electronic data Data access and use are management3 for data management management migrate data collection and management processes constantly monitored, procedures exists for the procedures for the HIS reporting from a paper are published and and data management national HIS. areclearly developed system to an electronic disseminated for the HIS. systems are updated and documented in a system, complete with A standard operating accordingly. Electronic nationally recognized necessary data security procedure and/or data use data transmission is the document. safeguards. A documented plan is in place to facilitate default method to move mechanism is in place for data use by the country data among information maintaining data quality and its stakeholders. systems. Dashboards 1+ throughout the data supply A data warehouse, displaying information chain. integrating data from from multiple sources all HIS subsystems are available todecision and allowing for data makers. triangulation and quality control, is fully functional and in use. HIS subsystems The country’s HIS mainly HIS data exchange Guidelines for compliance The government requires Most HIS subsystems consists of stand-alone is mainly facilitated with technical standards all HIS subsystems to are exchanging data program- specific by a single subsystem for HIS subsystem comply with the country’s electronically, according subsystems working in directly linked to other interoperability with the interoperability plan, to industry standards/ best silos, and addressing subsystems to enable national HIS have been including use of technical practices. only the asic information basic data exchange. disseminated. standards. needs (routine HIS, surveillance system, and An increasing number of 1+ human resources). Program- HIS subsystems areweb- specific parallel systems based and integrated exist. with the ISL following the national standards requirements. Operations and Operations and Maintenance for The country is receiving The country has the The operations and maintenance maintenance services for network and hardware technical support to build a capacity for strong maintenance services plan (for computer electronic systems are ad is a mix of reactive and strong in-country capacity in-country technical is continuously reviewed technology) hoc or non-existent. evolving preventive for computer technology maintenance. Computer and adapted to evolving procedures. maintenance. Standard operations and HIS interoperability operating procedures maintenance services are requirements, and follows exist that detail protocols part of the HIS plan or the industry-based standards. 2 for routine network and country’s strategic plan for Regular simulations are hardware maintenance.v health. A disaster recovery undertaken to increase the plan for digital HIS is in ability of technology staff place, and it meets best to respond to a disaster. practices.

3 Procedures on how data are captured, stored, analyzed, transmitted, and packaged for use across the data supply chain.

12 MEASURE Evaluation September 2019

Subdomain Domain Subdomain Level 1: Nascent Level 2: Emerging Level 3: Established Level 4: Institutionalized Level 5: Optimized Level Technology Communication The country has no An ICT infrastructure A national implementation All national offices and All or almost (>75%) all network: local reliable network assessment has been plan to meet the LAN and at least 50% of the the MOH’s national and area network connection to support conducted to determine WAN requirements in the subnational offices of the subnational offices and (LAN) and wide anational HIS. LAN and WAN country exists. A national MOH and health service health service providers area network requirements for the network maintenance plan providers have a strong have a reliable and robust (WAN) country’s HIS. exists to ensure high uptime, and reliable network network connection. A including procedures to connection to the various team dedicated to support The country is using recover from network failure. HIS network services. An connectivity exists and has 3+ mainly unreliable The country has started HIS-dedicated ICT and adequate financial, human, (2G, 3G or 4G) modems to implement a technical network support team is in and technology resources. to connect to the HIS solution to ensure permanent place. Industry-based standards services. connectivity to the HIS are followed. services.

Hardware The country has An ICT infrastructure Fifty percent (50%) or more Seventy-five percent (75%) The hardware meets limited/inadequate assessment has been of the MOH’s national and of the MOH‘s national and national and/or hardware (servers, done to identify the subnational offices have subnational offices have international specifications, user computers, hardware required at the required hardware, the required hardware. and a long-term plan (five printers, and national and subnational including back-up hardware. There is a back-up and years or more) is in place supportive levels. Less than 50% of recovery plan for the that details how to keep accessories) to the MOH’s national and national HIS. hardware up to date. 1+ support a national subnational offices have HIS. the required hardware (computers, printers, connecting devices, etc.). Domain maturity level: Technology 1

This publication was produced with the support of the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L-14-00004. MEASURE Evaluation is implemented by the Carolina Population Center, University of North Carolina at Chapel Hill in partnership with ICF International; John Snow, Inc.; Management Sciences for Health; Palladium; and Tulane University. Views expressed are not necessarily those of USAID or the United States government. FS-18-296