MyJustice Mid-Term Review
Final Report
Submitted to: British Council
23 March 2018
Integrity Somerset House, West Wing Strand London, WC2R 1LA T +44 (0) 207 759 1119 E [email protected] W www.integrityglobal.com
Acknowledgement
This report has been written by Integrity as part of its Learning & Evaluation work undertaken on behalf of the British Council in support of its MyJustice programme in Myanmar. This report does not necessarily reflect the views of each of the contributing partners in all its content. Any errors are the sole responsibility of the authors.
For further information, please email [email protected].
Table of Contents
Executive Summary ...... ii Acronyms ...... v Glossary of key terms...... vi 1 Introduction ...... 1 1.1 Review purpose and objectives ...... 1 1.2 MyJustice programme overview ...... 1 1.3 Methodology ...... 2 1.4 Review findings ...... 4 2 Relevance ...... 4 2.1 Understanding the justice needs of target groups ...... 5 2.2 Responsiveness to context changes ...... 7 2.3 Alignment of activities to the programme’s theory of change...... 11 3 Efficiency ...... 11 3.1 Factors supporting efficiency ...... 12 3.2 Factors limiting efficiency ...... 15 4 Effectiveness ...... 19 4.1 Supporting communities to increase knowledge of their rights and pathways to justice .. 20 4.2 Increasing the availability and quality of justice services ...... 22 Case Study: Yangon and Taunggyi Justice Centres ...... 25 4.3 Strengthening Community Based Dispute Resolution ...... 28 4.4 Using evidence and knowledge sharing to inform justice policy development ...... 29 5 Impact ...... 30 6 Sustainability ...... 34 6.1 Sustainability of the programme ...... 34 6.2 Sustainability of MyJustice partners ...... 35 7 Coherence and EU value added ...... 37 7.1 Coherence with EU priorities ...... 38 7.2 EU value add ...... 39 7.3 Other justice sector programming ...... 40 8 Conclusions and recommendations ...... 43 9 Annexes ...... 48 9.1 MyJustice Logical Framework ...... 48 9.2 Review Framework ...... 55 9.3 Mid-Term Review Terms of Reference...... 60 9.4 CV of evaluators ...... 66 9.5 List of organisations consulted ...... 79 9.6 List of key programme documents reviewed ...... 81
www.integrityglobal.com i
Executive Summary MyJustice programme The MyJustice programme (MyJustice) is a four-year European Union (EU)-funded programme implemented by the British Council. MyJustice has completed over two years of implementation in a complex context where communities have significant justice needs but also a deep distrust of the formal justice system, and quality justice services are difficult to access. MyJustice aims to enhance access to justice for these communities through four programme pillars (Result Areas) that broadly focus on: I) awareness raising activities; II) strengthening justice services; III) strengthening community-based dispute resolution mechanisms (CBDR); and IV) engaging with justice sector policy. MyJustice prioritises a learning and adaptation approach to implementation underpinned by three core principles: being people-centred, community-led and evidence-driven. Implementation is predominantly undertaken in partnership with local Civil Society Organisations (CSOs) and International Non-Government Organisations (INGOs) through grants or other contract mechanisms. Through these partnerships and some direct implementation, MyJustice delivers community-focused and responsive programming to ‘enable more people to have the knowledge, confidence and opportunities to have their disputes fairly and equitably resolved.’
Mid-Term Review This report consolidates the findings from the Mid-Term Review that took place in from January to March 2018. Integrity, the MyJustice Learning and Evaluation partner, conducted the review to provide evidence-based findings about MyJustice’s progress against desired results in its third year of programming. The review aims to highlight ‘what is working and what is not’ to inform planning and decision-making during the remainder of the programme period. The review is delivered against a framework of nineteen review questions that assess aspects of programme relevance, efficiency, effectiveness, impact, sustainability, coherence, and EU value add (see Annex 9.2 for questions). The findings are based on a systematic review and analysis of existing programme data and primary data collection.
Findings and recommendations MyJustice has adopted a flexible and contextually responsive programming approach that is both relevant and necessary given the complexity of the Myanmar justice sector context. The programme has developed and tested a range of tools and approaches, including the result areas strategic framework tool, to support staff to identify whether programme-level activities remain relevant and are achieving the desired effect. Where activities or approaches are not working, the program has processes to adapt and refocus resources to better promote the achievement of its objectives. One example is the design and implementation of the CSO Capacity Development Strategy in response to capacity issues among partners. Currently, MyJustice is on track to meet its output level results through its awareness raising activities, establishment of justice centres, and training of justice service providers. While the programme aims to support the most vulnerable, gender is not mainstreamed in the programme approach and limited attention is given to ensuring ethnic and religious minority justice needs are addressed. MyJustice has contracted a separate gender and social inclusion (GESI) assessment to review these gaps. The programme’s learning and adaptive approach is integral to ensuring programme relevance, efficiency and effectiveness. However, there is limited evidence that this approach has been adopted by local partners, who are constrained by limited technical capacity and knowledge regarding strategic adaptive approaches. The programme’s monitoring system captures some key data, but there are weaknesses that limit the effectiveness and
www.integrityglobal.com ii
efficiency of the system to track programme progress. There is limited data available for some indicators that measure knowledge, attitude and behaviour change of beneficiaries. As a result, progress towards higher-level objectives is difficult to ascertain at this point in the programme. While the theory of change remains relevant, the expected change pathways need some further consideration given lesson learning and tested assumptions. For instance, local partners need to strengthen their capacity to provide awareness raising activities that will impact target beneficiaries’ knowledge of legal rights and justice pathways. Mercy’s Corps community-based dispute resolution training is currently on hold due to the unexpected elections for ward and village tract administrators (Mercy Corps’ target trainees). While the expected degree of change might not be possible within the programme timeframe, there are indications that the programme can contribute to longer-term change with further consolidation and application of lesson learning.
Key findings
Relevance • The programme generally understands and responds to justice needs in target areas, but this could be strengthened through greater engagement at a partner-level, particularly regarding gender and inclusion needs. • The programme’s political economy engagement and adaptive management approaches are complementary and encourage well-informed and relevant responses to context changes. • Programme-wide understanding and commitment to a theory of change-based approach risks being diluted by large number of activities and partners, and rapid internal organisational growth. Efficiency • Key factors supporting efficient delivery include; reflective, considered and communicated theories of change and objectives; governance structure linking the programme with British Council stakeholders; and a management approach that enables strategic decision making based on learning. • Factors that limit efficiency include: unclear contracting processes at programme and partner level; partners’ challenges with reporting requirements, administrative processes and visibility requirements; and weaknesses in programme monitoring arrangements. Effectiveness • Lessons from partner-implemented community awareness raising activities has resulted in MyJustice adapting its approach to focus on critical partner capacity development to strengthen activity quality • Justice centres are reaching more townships and people, and justice service providers are being trained, but user satisfaction levels and improved capacity of trainees is challenging to measure • CBDR actors are being trained but knowledge and behaviour change will be challenging to evidence during the programme period
• Government buy-in to the MyJustice approach and research findings indicates progress in policy engagement and potential for influence Impact • While there has been progress within result areas, there is limited evidence of wider change to date • Corruption and community distrust of the justice system remain key barriers to achieving the programme’s desired impact • CSO capacity development activities indirectly contribute to sustainability but organisational development needs remain
www.integrityglobal.com iii
Sustainability • Future programme funding has yet to be secured and sustainability contingency planning is important given recent global political developments • Lesson learning and programme advocacy to government bodies are important strategies for promoting long-term sustainability of justice centres • CSO capacity development activities indirectly contribute to sustainability but organisational development needs remain Coherence and EU Value Add • MyJustice programme approach and objectives generally align with EU policies and priorities in Myanmar • The MyJustice-EU partnership encourages two-way sharing of expertise and opportunities exist for greater alignment and coordination with other EU/member states programmes • High-level justice sector coordination by MyJustice and the EU encourages complementarity of interventions and opportunities exist for MyJustice to support stronger coordination at a community- level
Key recommendations Based on these findings, the review team developed a series of recommendations that are presented throughout the report and consolidated in the conclusion. The following is a summary of the priority recommendations for immediate consideration by the programme: 1. Engage with the GESI assessment and work with partners to address and implement these findings at both programme and partner-levels. 2. Review existing partner engagement mechanisms (e.g. partner forums) and internal staffing roles and responsibilities (e.g. field offices) to promote more systematic engagement of partners and sub-partners regarding grassroots justice needs. 3. Conduct an internal review of the contracting process to ensure (to the extent possible) processes are aligning to appropriate due diligence requirements and clarify roles and responsibilities within this process. 4. Update the Monitoring, Evaluation and Learning (MEL) framework to ensure all targets are set, baseline data is defined, and responsibilities are clearly articulated across the team. 5. Prioritise monitoring of changes in knowledge, attitudes and practices of programme beneficiaries (community members, partners, service providers, and other justice service actors) to generate evidence on progress towards objectives 6. Undertake a participatory programme synergy mapping exercise to support MyJustice staff team and partners to understand the wider programme approach and theory, reflect on how resources are currently being used and to what effect, and assist to prioritise activities for Year 4 and inform a potential second iteration of the programme. 7. Continue to engage the EU and British Council in discussions regarding programme sustainability as a matter of urgency, including contingency planning in case future EU funding is unavailable. 8. Engage with all partners to support the development of financial sustainability strategies. 9. Discuss with the EU current and/or future opportunities for EU-funded programmes such as MyJustice and MyPol to better coordinate to achieve EU desired results. 10. Discuss with partners and CSOs opportunities and potential mechanisms for greater community-level coordination, including for example, piloting one or more mechanisms in specific locations.
www.integrityglobal.com iv
Acronyms PEA – Political Economy Analysis PRLP – Promoting Rule of Law Project A2J – Access to Justice ROM – Results-Orientated Monitoring CBDR – Community Based Dispute Resolution TJC – Taunggyi Justice Centre
CSO – Civil Society Organisation UAGO – Union Attorney General’s Office
DIHR – Danish Institute for Human Rights UK – United Kingdom
DoA – Description of Action ULAB – Union Legal Aid Body
EAOs – Ethnic Armed Organisations UNDP – United Nations Development Programme EU – European Union VFM – Value for Money FGD – Focus Group Discussion VSO – Voluntary Service Overseas GAD – General Administration Department WVTA – Ward or Village Tract Administrator GESI – Gender Equality and Social Inclusion YJC – Yangon Justice Centre IDA – Institute for Development Administration
IDLO – International Development Law Organisation
ILAM – Independent Lawyers Association of Myanmar
ILF – The International Legal Foundation
INGO – International Non-Governmental Organisation
JSCB – Joint Co-ordination Body for Rule of Law Centres and Justice Sector Affairs
KII – Key Informant Interview
MEL – Monitoring Evaluation and Learning
MJC – Mawlimyine Justice Centre
NGO – Non-Governmental Organisation
OCA – Organisational Capacity Assessment
OSCU – Office of the Supreme Court of the Union
PAGoDA – Pillar Assessed Grant or Delegation Agreement
PDIA – Problem Driven Iterative Adaptation
www.integrityglobal.com v
Glossary of key terms
Adaptive programming – A programme approach that puts learning at the centre and prioritises working in ways that are politically smart and locally led.
ASANA – A programme management tool (www.asana.com)
Beneficiary – A person who has benefitted from the actions of the programme.
Capacity-building/development – Strengthening the skills, competencies and abilities of people and communities.
Communities of practice – A group of people engaged in a process of collective learning in a shared endeavour.
Community based dispute resolution – Mechanisms, generally outside of the formal justice system, for resolving community-level individual and group disputes.
Disaggregated data – Information that has been broken down by characteristic of the source, such as gender, age or location.
Evaluation – A systematic determination of a subject's merit, worth and significance, using criteria governed by a set of standards.
Exclusion – The social disadvantage and relegation of individuals or groups to the fringe of society.
Gender dynamics – The relationships and interactions between girls, boys, men and women.
Programme governance – The structure of executive and management roles, program oversight functions and policies that define management principles and decision making.
Implementation – The process of putting a decision or plan into effect.
Indicators/proxy indicators – A means of measuring actual results against planned or expected results in terms of quantity, quality and timeliness.
Knowledge Attitudes and Practice Survey – Survey measuring Knowledge, Attitudes and Practices.
Justice Sector Coordination Bodies – Established under Notification 14/2017, the Union-level Joint Co-ordination Body for Rule of Law Centres and Justice Sector Affairs (JSCB) is chaired by the Union Attorney General and includes representatives of the executive, parliament, judiciary, academia, the human rights commission, private lawyers and civil society. It ensures co-ordination among justice sector interventions such as the Rule of Law Centres and justice centres. The JSCB has established state/regional-level co-ordination bodies, with a mandate to identify priority problems and solutions in the justice sector, engage with the public and contribute to the development of a national justice strategy.
Legal aid – The provision of free legal assistance to people otherwise unable to afford legal representation.
Logframe – A tool for improving the planning, implementation, management, monitoring and evaluation of projects. The logframe is a way of structuring the main elements in a project and highlighting the logical linkages between them.
www.integrityglobal.com vi
Metrics – A system or standard of measurement.
Milestones – Used to mark specific points along a project timeline. These points may signal anchors such as a project start and end date, a need for external review or input and budget checks.
Minorities – Culturally, ethnically, linguistically or racially distinct groups that coexist with but are subordinate to a more dominant group.
Monitoring, Evaluation and Learning Framework – Project document that defines all programme indicators, includes baseline information, and identifies sources of data and responsibilities for data collection.
Paralegal – A person trained in subsidiary legal matters but not fully qualified as a lawyer.
Policy – A course or principle of action adopted or proposed by an organization or individual.
Political economy analysis – The interaction of political and economic processes in a society: the distribution of power and wealth between different groups and individuals, and the processes that create, sustain and transform these relationships over time.
Pro bono – Work undertaken without charge, especially legal work for an indigent client.
Programme assumptions – A statement that is established as true for the purposes of implementing a programme.
Qualitative – Research that is primarily exploratory. It is used to gain an understanding of underlying reasons, opinions, and motivations.
Quantitative – Research that is used to quantify an issue by way of generating numerical data or data that can be transformed into usable statistics.
Result areas – The pillars of the MyJustice programme that represent the four conditions for change outlined in the theory of change
Result areas strategic framework – A bespoke MyJustice programme management tool that provides a theory of action for how each result area specifically contributes to achieving the overall programme objective.
Justice service providers – Individuals, such as lawyers, paralegals and officials, who provide justice services such as legal representation, legal advice, or decision making.
Stakeholders – Individuals or group that have a direct or indirect interest in the issue being explored.
Theory of change - A specific type of methodology for planning, participation, and evaluation that is used to promote social change. Theory of Change defines long-term goals and then maps backward to identify necessary preconditions.
Vulnerability – The diminished capacity of a group or individual to realise their basic rights and access justice for violations of those rights.
www.integrityglobal.com vii
1 Introduction Since November 2016, Integrity has worked with MyJustice as an external Learning and Evaluation partner to help strengthen programme performance, support accountability to beneficiaries, and provide evidence of ‘what is working and why’. As part of its evaluation activities, Integrity conducted a formative evaluation in the form of a Mid-Term Review (the review) from January to March 2018. This report presents the key findings of the review process to inform the programme’s overall learning about what works and what does not. It provides a series of recommendations designed to support the programme’s Year 4 work planning and discussions concerning a second iteration of the programme beyond July 2019.
1.1 Review purpose and objectives The review has two primary objectives: • To provide evidence-based findings about MyJustice’s progress against the results outlined in the programme’s logical framework (revised December 2017); and • To explain how and why progress towards results has or has not been made.
This review is a process-focused exercise, designed to complement but not duplicate MyJustice’s own data collection efforts. The report explores the areas where results have been slower to materialise than anticipated during the programme design. It also identifies specific lessons learned about access to justice (A2J) programming in Myanmar from the MyJustice experience. The report serves as the foundation for the programme’s final evaluation in 2019. This report expands the body of existing evidence generated through the programme’s wider Monitoring, Evaluation & Learning (MEL) framework. It builds on Integrity’s Annual Learning Review report submitted in July 2017. That report produced a series of learning and adaptation- related findings that served as a guide for the programme as it moved into its third year of implementation.
1.2 MyJustice programme overview MyJustice is a four-year A2J programme implemented by the Desired change: British Council in Myanmar. The €20-million programme is People have the knowledge, funded through the European Union’s (EU) Pillar Assessed “confidence and opportunities to Grant or Delegation Agreement (PAGoDA) modality.1 have their disputes fairly and equitably resolved Commencing in August 2015 with a nine-month inception ” phase, the programme is currently in its third year of MyJustice Theory of implementation. The programme is scheduled to end in July Change Analysis 2019. MyJustice prioritises a learning and adaptation approach to implementation that is underpinned by three core principles: being people-centred, community-led and evidence-driven. The programme aims to contribute to: a) improving access to justice and legal aid for the poor and vulnerable;
1 Under the PAGoDA modality, the British Council, as a pillar assessed organisation, was able to directly negotiate the programme agreement with the EU (as opposed to competitively winning a tender) and implements its own policies and procedures in managing and controlling the programme.
www.integrityglobal.com 1
b) developing legal capacity of justice sector professionals; and c) strengthening selected rule of law institutions to better fulfil their mandates.
MyJustice is predicated on a theory of change that: ‘if people have greater awareness of their rights and pathways to access justice and these options are supported to be more widely available, more inclusive, accountable and of a higher standard, reflecting people’s justice needs, then the options and services will become more responsive to the needs of the people; and more people will be able to address their grievances and access justice.’ This is based on an informed expectation that ‘with increased knowledge, confidence and opportunities, people will be able to have their disputes fairly and equitably resolved.’2 The programme seeks to facilitate change through work across four result areas described at Table 1.
Table 1: MyJustice Result Areas
Result Focus Expected change Area Increased capacity within communities to understand and apply 1 Awareness raising their rights. Justice services are more widely available, of higher standard, and 2 Justice services increasingly meet the needs of communities and their residents. Community based dispute CBDR mechanisms increasingly utilise fair approaches for dispute 3 resolution (CBDR) resolution within the communities they serve. Justice policy Increased evidence and knowledge sharing to inform justice policy 4 engagement development and implementation.
MyJustice engages partners through grants and service contracts to implement activities across the four result areas. Partners include national civil society organisations (CSOs) and international non-governmental organisations (INGOs). The MyJustice team in Yangon provides programme direction and management and undertakes some direct implementation, particularly under Result Area 4 (justice policy engagement). The programme recently established two field offices in Mandalay and Mawlimyine to engage with partners operating in those locations.
1.3 Methodology Review questions and design process Integrity designed the review methodology and questions in close collaboration with the MyJustice team and in consultation with the EU Myanmar Delegation. The Inception Report (revised November 2017) and annexes set out the scope of the review. The review framework is grounded in the OECD Development Assistance Committee (OECD/DAC) and EU evaluation criteria. It is framed across key questions and sub-questions in seven thematic areas: 1. Relevance 2. Efficiency 3. Effectiveness 4. Impact
2 MyJustice Theory of Change Analysis
www.integrityglobal.com 2
5. Sustainability 6. Coherence 7. EU Value-Add
The specific review questions are included in Annex 9.2. The questions went through various iterations and were finalised in January 2018 before the primary data collection began. The questions focus the review on assessing MyJustice’s processes to demonstrate how these impact on the programme’s progress towards its objectives. Review methods and tools The review involved both primary and secondary data Figure 1: Number of key documents collection. The review team examined 135 documents, 73 External of which were classified as key reference documents. 8 The review team collected primary data from 110 respondents through 50 Key Informant Interviews (KIIs) and 13 Focus Group Discussions (FGDs). The inception Partner report outlined the sample of respondents. During data 21 MJ collection, the review team made some slight adaptations 44 to the sample considering the availability of key individuals. Primary data collection did not include interviews or surveys with community members or justice sector actors, such as judges and police, due to the agreed scope and available review resources. The process-focus of the review relied particularly on interviews with grantees, partners and sub- partners as well as MyJustice staff, other A2J implementers and donors, and Myanmar government stakeholders. The review team conducted interviews in English and/or Myanmar language in Yangon, Taunggyi, Mawlimyine, Naypyitaw and remotely (by Skype) between 24 January and 15 February 2018.
Figure 2: MTR Respondents
Grantees, partners and sub-partners MyJustice Staff Myanmar Government Other foreign donors Other A2J implementers British Council Staff EU
0 5 10 15 20 25 30 35 40
FGDs KIIs Observations
The review team coded and consolidated all data using Nvivo software and examined the guiding questions using a combination of comparative and adapted contribution analysis. The team conducted analysis and report writing from 20 February until 23 March 2018.
Challenges and mitigations The following is an overview of the key challenges and mitigation actions.
www.integrityglobal.com 3
Table 2: Review challenges and mitigations
Challenge Mitigation Not enough time has passed As some interventions are very recent or on hold, we were not able to to assess intervention assess intervention implementation in some instances. Impact analysis implementation and/or impact reviewed the evidence underpinning the change pathways and assumptions to identify gaps in intervention implementation. Data unavailable on some Some data has not yet been collected, specifically regarding knowledge, programme level indicators attitude and behaviour change. This limited the extent to which we could assess change and progress. However, questions were included in the questionnaires to ask for any initial evidence of change, to assess the extent to which partners are seeing indications of expected change. Key stakeholder unavailable The majority of intended interviews were conducted due to forward for interviews planning and support from the MyJustice team. However, some stakeholders were unavailable during the data collection period. In these instances, alternative stakeholders were identified. Partners unaware of the Some partners were unclear as to the purpose of the review. The team purpose of the review ensured the objectives were discussed before the interview began and the open interview structure facilitated partners to ask questions. Questionnaires were developed and discussed across the team to identify strategies to encourage open discussion. Generally, respondents were open to sharing both positive and challenging aspects of the programme. The justice sector in Myanmar We conducted more key informant interviews conducted than FGDs. is sensitive and often These interviews facilitated more candid discussion. Some partners did politicised. Stakeholders not understand the purpose of the review, which required our team to unwilling to talk about issues spend time explaining the process and purpose to facilitate an open openly discussion. Confidentiality was maintained throughout the review with interview notes kept within the team only and sources not attributed within the final report.
1.4 Review findings This report presents the review findings organised under each of the seven thematic areas. Each thematic section describes the purpose of the theme, highlights key findings, and presents recommendations designed to support MyJustice planning and implementation during the remaining programme period.
2 Relevance Overview and key findings This section assesses the extent to which the programme is suited to the justice priorities and needs of its target communities and, in particular, the poor and vulnerable. It examines the programme’s responsiveness to the operational context. Finally, it reviews the consistency of programme activities to the overall goal and theory of change.
www.integrityglobal.com 4
Key findings • The programme generally understands and responds to justice needs in target areas, but this could be strengthened through greater engagement at a partner-level, particularly regarding gender and inclusion needs. • The programme’s political economy engagement and adaptive management approaches are complementary and encourage well-informed and relevant responses to context changes. • Programme-wide understanding and commitment to a theory of change-based approach risks being diluted by large number of activities and partners, and rapid internal organisational growth.
2.1 Understanding the justice needs of target groups MyJustice engages partners to implement activities that aim to improve access to justice for the poor and vulnerable either by targeting vulnerable groups directly and/or targeting justice service providers (i.e. legal aid lawyers and paralegals, General Administration Department (GAD) officials, Ward or Village Vulnerable people are defined as Tract Administrators (WVTAs), and community leaders) to “those who experience power provide more responsive, higher quality justice services to disparity, poverty, exclusion or discrimination and lack available the poor and vulnerable. MyJustice partners support a wide avenues through which to seek range of vulnerable groups, including indigent criminal redress for a rights violation. accused, women, farmers, migrant workers, LGBT and ” MyJustice workshop, early 2017 others.
Finding 1: MyJustice considers national-level justice needs but has more limited understanding of localised or specific needs, including gender and inclusion needs At the programme-level, MyJustice generates information on the justice sector and evidence on justice needs across its target communities through a range of approaches.3 The programme identifies opportunities for activities that further strengthen its understanding of justice needs, including, for example, through its engagement with the government-formed state and regional- level JSCBs, or the justice scorecards partner activity. The programme’s level of engagement with partners to better understand community-level justice needs is varied. Some sub-partners perceive that their knowledge of grassroots justice needs, and potential responses, is not systematically informing programme knowledge. Several MyJustice staff also suggested that the programme still has a weak understanding of justice issues at a community-level. Systematic and in-depth engagement at a community-level is hindered by: i) The multi-level grant structure – Working predominantly through partners and sub- partners means MyJustice is one or two steps removed from communities. Sub-partners report to partners, not to MyJustice directly, so information may be filtered out or lost in
3 These include a political economy analysis, commissioned research pieces (such as the access to justice research carried out by ODI and Saferworld and the rapid context briefs conducted by Integrity), the recent KAP baseline and national surveys, as well as drawing from available external research and analysis.
www.integrityglobal.com 5
the tiered reporting structure. There is no dedicated mechanism for discussing justice needs with partners and sub-partners. ii) The large number of partners – As the number of partners (approximately 40 at the time of the review) increases, MyJustice’s staff capacity to engage deeply with them all is reduced. No single staff member can have knowledge across all partners and their activities to identify all possible geographical or substantive programme connections. iii) The internal result area structure – Relevant partner data and information under one result area is not always informing understandings within another result area, i.e. linkages between some partners and activities across result areas are being missed which reduce their effectiveness and potential impact.
MyJustice is taking some steps to try and mitigate these factors, see Figure 3.
Figure 3: Programme-level and partner-level knowledge generation
According to MyJustice staff and the available evidence, the programme has limited understanding of minority groups and their specific justice needs. Some partners are seeking to address the gender gap based on their own internal system requirements. However, other partners do not have these types of internal checks. MyJustice has recently initiated several activities designed to increase programme understanding of gender dynamics and/or to strengthen partner capacity to understand and advocate on gender justice issues, including contracting the Danish Institute for Human Rights (DIHR) to conduct a Gender Equality and Social Inclusion (GESI) assessment; and initiating a ‘women and the law’ policy activity under the CSO Capacity Development Strategy (the CSO Strategy). MyJustice staff indicated the programme is considering if and how to engage Ethnic Armed Organisations in the future and believe data from the nationwide Knowledge, Attitudes and Practices (KAP) survey and
www.integrityglobal.com 6
baseline survey, as well as findings from new partner projects will contribute to programme learning on this issue. • Recommendation 1: Engage with the GESI assessment and work with partners to address and implement these findings at both programme and partner-levels. • Recommendation 2: Review existing partner engagement mechanisms (e.g. partner forums) and internal staffing roles and responsibilities (e.g. field offices) to promote more systematic engagement of partners and sub-partners regarding grassroots justice needs.
Finding 2: Programme activities generally address identified justice needs of the target group, but it is unclear how emerging data on justice needs informs programme adaptation. There is some ambiguity about when and how evidence of justice needs informs activity design and adaptation. Some research outputs were completed after interventions had already been planned or initiated. These have informed subsequent policy engagement activities, but there is limited evidence of how they have informed programme adaptations. Several MyJustice staff noted that the KAP and baseline survey data still needs to be fully analysed to inform specific programming decisions. Programme interventions under Result Areas 1, 2, and 3 largely align with justice needs identified through programme and external research, partner engagement, and the KAP survey and baseline data. A range of external and partner respondents identified one specific justice issue that the programme does not directly address, namely addressing the issue of police abuse. Other respondents also emphasised the key role of the police for ensuring access to justice at the community-level and suggested MyJustice should better engage this stakeholder group in order to promote systemic change within the justice system. Police are currently involved in some limited MyJustice activities, such as a roundtable discussion in Taunggyi with the justice centre. Learnings from these engagements, and the EU MyPol programme, could inform the design of networking and cross learning opportunities with the police in a future iteration of the programme to promote a more systematic approach to justice sector change (see section 7.2 for further discussion and recommendations). MyJustice policy engagement interventions have so far primarily focused on relationship building and general awareness raising of access to justice issues to government. These interventions are not specifically promoting the justice needs of the poor and vulnerable, with the exception of some limited engagement with the Union Legal Aid Body (ULAB) and Office of the Supreme Court of the Union (OSCU) on the Legal Aid Law (2016). The programme’s engagement with the state/regional JSCBs provides an opportunity for MyJustice to more clearly link its policy work to the target group. Other opportunities include the recent women and law policy activity under Result Area 4 that supports partners to strengthen their policy engagement skills using the theme of women’s access to justice. Another example is the Result Area 2 team’s targeted analysis of justice centre case data. • Recommendation 3: Undertake analysis of Knowledge, Attitudes and Practice and baseline data to inform programme adaptation and to ensure the programme remains responsive to the justice needs of the poor and vulnerable. • Recommendation 4: Consult with partners and other relevant stakeholders regarding options for engaging the police in a future iteration of the MyJustice programme.
2.2 Responsiveness to context changes
www.integrityglobal.com 7
Finding 3: MyJustice responds to changes in access to justice-related context dynamics to ensure continued relevance of interventions and approaches. The programme’s understanding of the Myanmar context and shifting political and social dynamics, demonstrated through its regular reporting, is well informed and aligns with the general perspectives and analysis of other key stakeholders. One aspect that distinguishes MyJustice’s adaptive approach from merely ‘good programming’ is its prioritisation of understanding power and political dynamics to inform programme design and adaptations.4 The programme has adopted a political economy engagement approach that focuses on regular identification and analysis of context changes that have the potential to impact the programme.5 MyJustice applies a range of tools in implementing this approach at a programme-level.6 This has enabled MyJustice to respond to changes in context in a timely and appropriate manner, which in turn has supported the programme’s adaptive approach. Figure 4 provides a sample of critical context changes and subsequent programme responses with a specific focus on events occurring after the Annual Learning Review exercise conducted by Integrity in early 2017.
4 Blog, ‘Adaptive Management looks like it is here to stay. Here’s why that matters’. 8 December 2016. www/oxfamblogs.org/fp2p/adaptive-management-looks-like-its-hear-to-stay-heres-why-that-matters 5 Political Economy Engagement is the process of constantly learning about, and responding to, political contexts. As a process, it focuses on how critical information is collectively distilled within the programme to ensure programme staff and implementers (such as the grantees) better understand and can adapt to the complex environments within which they conduct their work and implement programmes. See for example, https://oxfamblogs.org/fp2p/what-are-governance-advisers-missing-with-political-economy-analysis-how-can- they-do-better/. 6 These include: Hiring of politically astute staff; regular political economy/context debriefs by national staff during team meetings; targeted, rapid research to inform programme decisions; regular engagement with key stakeholders, for example the union and state/region JSCBs; capturing ‘critical moments’ in result area strategic frameworks and ASANA; facilitated quarterly reflection sessions; use of Slack (communications app) for YJC and MyJustice; media monitoring.
www.integrityglobal.com 8
Figure 4: Context events and programme response
www.integrityglobal.com 9
www.integrityglobal.com 10
2.3 Alignment of activities to the programme’s theory of change Finding 4: The programme regularly reviews interventions to ensure consistency with its theory of change, but opportunities exist for greater alignment. Both the Annual Learning Review and the current review exercise identified that key MyJustice and external stakeholders perceive the current programme and its activities to be largely consistent with the overall objectives and high-level theory of change. The programme reviews its interventions against the theory of change and programme assumptions using a range of tools.7 Result area leads regularly update the online programme management tool, ASANA, but, as with most learning-focused programmes, face a challenge of finding the right balance between capturing too little or too much information. They also use the result area strategic frameworks that articulate the theory of ‘action’ i.e. how activities under each result area connect to the high-level theory of change. This tool is valuable for enabling the programme to test assumptions, re-examine change pathways and make adaptations regularly. This is particularly useful for enabling an adaptive approach especially when the programme’s overall logframe can only be refined annually. The programme has shifted some activities, such as the KAP Survey and justice scorecards, from one result area to another to reflect more accurately the contribution these activities make to the theory of change. The review team observed that the English language training activity is not clearly aligned to its current result area. While the training for the Union Attorney General’s Office (UAGO) has achieved some results (see section 4.2), it is unclear how the activity is expected to contribute to improved justice services at the community level, which is the focus of activities under Result Area 2. MyJustice staff suggested that the activity has more significance for the policy engagement work under Result Area 4. • Recommendation 5: Transfer the UAGO English Language Training to Result Area 4 and explore with the teacher how the activity could more explicitly complement policy engagement efforts.8
3 Efficiency Overview and Key Findings This section examines the extent to which internal structures and processes have supported or acted as a barrier to the programme’s implementation and progress towards its stated objectives.9 This is not a detailed compliance review of the grants mechanism or a quantitative value for money (VfM) assessment. The section provides a snapshot of how the programme is currently operating to identify constraints to efficiency that can be addressed in the remainder period of the programme.10
7 Tools include result area strategic frameworks, ASANA, facilitated reflection sessions, quarterly reporting, annual revisions of the logframe, technical support from the Learning and Evaluation partner, and advisory support from the Programme Advisory Committee. 8 One example provided was using more programme research outputs in student exercises and having MyJustice team members engage more with the students during classes. 9 This analysis approach is adapted from the framework established in the EU’s ‘Value for money of EU programming funding in the field of democracy and rule of law’ study: http://www.europarl.europa.eu/RegData/etudes/STUD/2017/572705/IPOL_STU(2017)572705_EN.pdf 10 In examining factors enabling and constraining efficiency, the review examines the extent to which factors support or hinder the programme’s ability to deliver its workplan, reach milestones, implement on time, and within budget.
www.integrityglobal.com 11
Key findings • Key factors supporting efficient delivery include; reflective, considered and communicated theories of change and objectives; governance structure linking the programme with British Council stakeholders; and a management approach that enables strategic decision making based on learning. • Factors that limit efficiency include: unclear contracting processes at programme and partner level; partners’ challenges with reporting requirements, administrative processes and visibility requirements; and weaknesses in programme monitoring arrangements.
3.1 Factors supporting efficiency Finding 5: The programme articulates its objectives and approach, which are key requirements to enable efficiency. One key aspect of efficient programming is ensuring that the objectives of the programme are clear across the team and that the team understands how they will be met. This supports efficient delivery as it ensures clarity across the programme about what goals the activities and resources are contributing to, and how. MyJustice staff members can then make decisions in a timely and strategic way. The programme objectives are clearly articulated in the theory of change and result area strategic frameworks. Partners, with the support of MyJustice, have articulated objectives and programme logic in their own logical frameworks to align with the programme objectives. The programme also has tools and processes in place, such as the result area strategic framework and partner reports, to enable efficient decision making based on the programme’s clearly communicated objectives and approach.
Finding 6: MyJustice team and governance arrangements contribute to relevant and responsive programming decisions The programme is governed through an internal project board that meets monthly.11 Its members perceive the board to be useful, particularly for reviewing risk and mitigation strategies. The programme is supported by British Council offices in Myanmar, Brussels and the United Kingdom, each with specific support functions. Several external and programme respondents highlighted that the composition of the MyJustice team, including staff with a range of expertise and sound understanding of the Myanmar context, is an important factor of the programme’s success to date.
Finding 7: Communication activities increase programme reach to stakeholders enabling greater understanding of programme objectives and key messages The programme’s communications team publicises MyJustice’s work to donors and external stakeholders. The team uses various communications platforms to engage the public on access to justice issues. MyJustice’s communications activities also seek to support the programme’s
11This board includes the Team Lead, Project Manager, as well as the BC Country Director, BC Director of Partnerships and Contracts, MyJustice Finance Manager and BC Business Delivery Manager.
www.integrityglobal.com 12
achievement of its objectives. The communications team contributes to awareness raising and policy engagement work through, for example, the media campaign and packaging research and data points to present to government officials. Coverage of the justice centre openings were used as opportunities to showcase the programme achievements but also to disseminate key information to external stakeholders about legal aid and justice centres.12 The communications teams can play an integral role in identifying methods, tools, and processes to expand programme reach and impact and promote the goals of changing knowledge, attitudes and behaviours. There are opportunities for this role to be more clearly articulated and tracked within programme documentation. To date, partners understand the role of the communications team as being to share programme results externally and to set visibility requirements (see below). Given the aim of the communications team to contribute to programme change, greater consideration could be given to articulating and tracking this contribution across the result areas. • Recommendation 6: Adapt the result area strategic framework tool to include a section for ‘Communications’ that articulates how communications activities support achievement of the result area and programme goals.
Finding 8: Learning and adaptation has enabled strategic decision making, contributing to more efficient programming Efficient programming relies on regular feedback loops to ensure evidence of what’s working and what is not working can inform programming decisions and ensure efficient use of resources. From the design phase through to implementation, MyJustice has committed to an approach that is driven by learning and adaptation. Figure 5 presents this approach, its components and programme tools, and presents two activity examples to illustrate how learning and adaptation at the programme-level has led to key programmatic decisions affecting resource allocation. In the first example, the programme identified challenges with partner awareness raising activities and on reflection, chose to reallocate resources to strengthen the capacity of partners (instead of continuing to issue more grants for interventions with quality issues). In the second example, the programme reflected on the learning from the pilot training of WVTAs, confirmed the relevancy of the activity to the theory of change, and elected to continue the activity without significant change in order to gather more evidence of the effectiveness of the pilot approach.
12 The review did not include an assessment of the impact of the press coverage on the programme.
www.integrityglobal.com 13
Figure 5: Application of MyJustice learning and adaptation approach in two examples
www.integrityglobal.com 14
3.2 Factors limiting efficiency Finding 9: Contracting procedures can result in protracted processes, affecting timely engagement of partners and interventions The MyJustice implementation model places significant reliance on issuing contracts to organisations and service providers to deliver activities across the four result areas and in specific technical areas (such as monitoring, evaluation and gender analysis). The programme decided not to launch a second round of grants13 following an internal review process in March 2017. Key factors in this decision included partners’ implementation capacity and MyJustice’s own internal capacity to manage another grants round. MyJustice engages new partners through limited calls and direct awards of either grants or service contracts. Each award type has different financial and reporting requirements. Some programme stakeholders highlighted the benefits of this contracting approach, which allows MyJustice to identify potential partners and engage them directly to design and deliver relevant pieces of work. However, the relatively high number of direct awards has created uncertainty among some internal programme stakeholders about due diligence and sign-off authority. For instance, views differ on the appropriateness of grants versus service contracts and this issue requires further clarification and communication within the MyJustice and British Council teams. The lack of clarity has caused some delays particularly when British Council regional or headquarters staff are required to sign-off on direct award contracts and question the programme’s choice of award type. • Recommendation 7: Conduct an internal review of the contracting process to ensure (to the extent possible) processes are aligning to appropriate due diligence requirements and clarify roles and responsibilities within this process.
Finding 10: Partners find reporting requirements are burdensome and take up significant staff resources and time Partners submit technical and financial reports to MyJustice on a quarterly basis. Partner perspectives on this mechanism are mixed, but most agree that the reporting requirements are heavy and use up significant time and human resources. Similarly, MyJustice staff expressed concern as to whether some local partners have the organisational capabilities and/or internal resources to manage contracts of the current financial size, and the reporting processes.
Table 3 presents select examples of partner feedback regarding the reporting system, an assessment of the issue based on the review process and presents potential refinements to the system that could be made by the programme.
13 Between June 2016 and March 2017 MyJustice conducted its first round of grant-making, entering into 12 partnerships to work across 25 townships in Mon and Yangon and deliver awareness raising, paralegal support, advocacy, pilot community dispute resolution, and provide direct legal representation.
www.integrityglobal.com 15
Table 3: Sample of partner feedback on reporting system and recommendations
Sample of partner Review assessment Recommendations feedback Quarterly report A strength of the report format is that it Review the report format with format is complicated includes space for reflection on learning and partners to clarify its purpose and needs project adaptations. and reach common agreement simplification Partners do not have a clear understanding of on refinements for greater how their learning is used at the programme- efficiency. level. MyJustice provides a The programme’s engagement with partners is lot of separate evidenced in part through direct feedback on feedback on reports partner reports. However, multiple in-text Synthesise written feedback into which takes a lot of comments overwhelm partners who reportedly concise key issues requiring time to address struggle to answer and address each point. partners response. ‘ Providing comprehensive feedback for every
partner is time-consuming for MyJustice staff. Identify opportunities provide in- Communication over While email provides a useful written account person feedback and mentoring email can be of feedback and required actions, partners support to improve reporting unwieldy and English (specifically local CSOs) struggle with the systems. language is amount of feedback in English.14 problematic Budgets are very The feedback was highlighted by both INGOs Consider alternative budget detailed and strict and local CSOs. models for subsequent funding. The review did not examine partner budgets. The budget requirements align with the guidance outlined in the grants agreement.
• Recommendation 8: Clarify the information that is the most critical for the programme in terms of partner reporting. Inform partners about and focus MyJustice feedback on this critical information. • Recommendation 9: Review partner reporting formats with MyJustice programme and technical teams and partners to develop a more streamlined and efficient format for future grants under the current programme and/or in a second iteration of the programme.
Finding 11: MyJustice administrative processes are limiting timely and responsive adaptation by partners MyJustice has developed a change request form as a mechanism to support partners to adapt their activities based on learnings from implementation. This process is fundamental to MyJustice’s overall learning and adaptation approach. Many partners indicated a willingness to apply more adaptive approaches in their work but perceive the current grants structure (requiring sub-partners to report through partners) and the change request form to be inhibitive. Currently change requests tend to focus on budget and operational matters, not substantive programming. Some partners noted they are reluctant to submit substantive change requests due to an anticipated burdensome process or expected resistance from MyJustice. Several MyJustice staff reported that the change request process puts pressure on the programme team who need to adapt contracts, forecasts and processes to account for these changes. MyJustice is largely aware that these processes are
14 Note that this was also apparent at TJC where the lawyers working at the centre typically receive feedback from IBJ over email rather than in person.
www.integrityglobal.com 16
considered taxing and are limiting partner-level adaptation. The programme team is currently reviewing the change process and looking for ways to further ease the burden while also maintaining controls. The review did not include a detailed examination of the extent to which partners are adapting or the impact of adaptation, or lack of, on programming and efficiency. However, MyJustice staff noted they face challenges with encouraging learning and adaptation at the partner-level, due to the large number of partners, the limited resources to engage these partners more, the low capacity and lack of processes among partners to learn and adapt independently, and the MyJustice administrative systems that can inhibit adaptation. • Recommendation 10: Review the change request system in consultation with partners, programme and technical teams, to address perceived and actual barriers to partner-level activity adaptation.
Finding 12: Programme resources are spread thinly across multiple ‘small bets’ and consolidation of activity-based learning is needed to inform future programme decisions In the dynamic and fluid environment of Myanmar, MyJustice’s approach has been one of continuous monitoring and analysis of the context (see section 2.2), adaptation of its strategies and activities, and capitalising on emerging windows of opportunities, such as the advisory role to the Union-level JSCB. The programme was based on the theory of seeding many small projects (small bets) by using multiple strategies and piloting interventions ‘to see what works.’ This initial testing phase enables the programme to identify and prioritise those interventions that are most effective and can be expanded. While management and some long-term MyJustice staff understand this approach and the overall programme logic (i.e. how all the pieces intend to fit together to create change), findings from the Annual Learning Review and the review suggest this understanding is not shared by partners and some other staff. This lack of understanding impacts the capacity of staff and partners to fully engage in the programme approach – with its particular focus on piloting activities in a ‘safe-to-fail’ environment, encouraging learning from both what works and what does not, and seeking to be community-driven (not donor-led). Simultaneously, at the midway point of the programme, MyJustice staff acknowledge they have limited capacity to continue to seize new opportunities or explore more small bets. Several respondents reflected that the programme needs to be able to prioritise where it places its current financial and staff resources. Others emphasised that by more deeply engaging in existing activities, as opposed to continuing to seize new opportunities, the programme can maximise its learning to inform the next planning phase. • Recommendation 11: Undertake a participatory programme synergy mapping exercise to support MyJustice staff team and partners to understand the wider programme approach and theory, reflect on how resources are currently being used and to what effect, and assist to prioritise activities for Year 4 and inform a potential second iteration of the programme.15
15 A synergy map exercise visualises not only what the programme is doing, where and with who, but also how all the various elements of the programme and the Myanmar context are intertwined - the assumptions, the power and political dynamics – to support or conflict with the programme’s goals. It is a tool for seeing where resources are being invested, where the greatest impact is being had and enables the programme to prioritise or reprioritise activities. Undertaken in collaboration with staff and partners, it can contribute to creating greater understanding of the strategic whole and provide guidance not only for the focus of interventions for the remaining programme period but also a second iteration of the programme. The exercise can be done with pen and paper or using an online platform like https://kumu.io/about.
www.integrityglobal.com 17
Finding 13: Visibility requirements not well understood among partners, causing some delays and tension Among partners, there was limited feedback on MyJustice’s communications processes and how these affect their day-to-day work.16 Partners feedback focused specifically on their engagement with MyJustice on logos and visibility guidelines. INGO partners in particular, but not exclusively, perceived these guidelines to be unclear and confusing, sometimes leading to delays in implementation of project activities.
The branding guidelines they require us to use are very complex. Using the documents [is] very “difficult. Sometimes they encourage us to put the logos on everything for every activity. On the other hand, in the MyJustice forum, they didn’t put partner organisation logos anywhere – this is unfair. ” MyJustice partner
Finding 14: Lack of clarity regarding MyJustice staff roles and responsibilities limits efficient engagement across teams At the programme level, the roles and responsibilities of staff are defined in their employment terms of references. Nevertheless, members of staff noted that internal management structures are not always clear. For some, this provides useful space to think and work across teams to implement effectively. But others perceive the lack of clarity more negatively. MyJustice recently established field offices in Mandalay and Mawlimyine. The engagement of field teams is intended to extend the support and reach of the MyJustice programme to partners in the target states/regions. However, it remains unclear how the partners and MyJustice programme engage with these team members, and vice versa. The field officers were not included in respondents sample for the review and their roles and contribution to programme activities were rarely mentioned in interviews with other staff. • Recommendation 12: Identify how best to integrate and utilise the field officers in promoting programme objectives, particularly in relation to increased engagement with partners and sub-partners.
Finding 15: The programme has developed a monitoring framework, but some internal weaknesses limit the effectiveness and efficiency of monitoring activities MyJustice has monitoring and evaluation arrangements in place to enable programmatic learning, track progress, and ensure programme accountability. In Year 2, MyJustice conducted a baseline survey in its target areas to establish baseline data against key indicators. The programme has developed a MEL framework, which is regularly updated, to explain how each indicator is defined and monitored; to establish the programme target; and to identify team responsibilities for data collection. While the MEL framework is a live document that is routinely updated, the review identified some gaps that limit the effectiveness and efficiency of the framework in enabling clear monitoring processes. These gaps include: • Targets: Targets have not been established across all indicators. This limits the programme’s ability to track progress.
16 The communications team works with RA teams to engage partners, which could be a reason why partners did not speak directly to how communications work is supporting their work.
www.integrityglobal.com 18
• Baseline: The MEL framework lacks detail on the source of some baseline figures. Clarity on the source and sample of the baseline is important to ensure endline figures are comparable. • Responsibilities: Monitoring responsibilities are spread across MyJustice staff, specifically the result area teams. However, the MEL framework does not clarify what the actual roles and specific responsibilities entail (such as spot checks, actual data collection, etc.). MyJustice monitoring data relies primarily on partner reports that consist of data against specific indicators and contextual information. For the programme to be able to rely on the validity of this data, partners need to have structures and processes in place to ensure the correct data is being collected and can be validated. This review did not include an in-depth assessment of partners’ monitoring systems. However, there were indications that some partners struggle with implementing rigorous monitoring requirements: These include: • Limited understanding on how to measure more qualitative indicators, such as ‘quality’ and ‘increased capacity’. • Perception that existing processes are already burdensome and time-consuming. • Some risks with double counting, particularly in areas with multiple interventions.
MyJustice is aware of these issues and is in the process of engaging a consultant to support partners in defining and conducting qualitative monitoring. The Result Area 2 team is also engaged with the justice centres to identify what monitoring data is needed and to establish systems for data collection. • Recommendations 13: Strengthen the MEL framework by: a. Setting targets across all indicators to ensure progress can be tracked for both accountability and learning processes. b. Defining baseline data sources to ensure comparable endline data can be gathered. c. Establishing whether the final evaluation exercise is intended to contribute to endline data collection, and if so ensure the exercise is sufficient resourced. d. Defining the monitoring responsibilities of MyJustice staff members. e. Ensuring the MEL team undertakes systematic spot checks of partner data to check validity.
4 Effectiveness Overview and Key Findings This section examines whether the results of MyJustice activities are likely to lead to the achievement of its objectives in the programme’s remaining time. It also assesses the extent to which the programme’s monitoring systems are set up to adequately capture data against the key results. MyJustice’s logical framework (logframe) outlines the programme’s intended results and includes targets and baseline data against which to measure progress. It does not align targets to specific time-bound milestones, which enables the programme to regularly adapt activities and timelines while remaining focused on the ultimate target. Given the programme’s adaptive nature, this is both a practical and appropriate approach. In the absence of milestone targets, however, the review examines the likelihood that objectives will be achieved by reviewing and assessing: a) The baselines of key indicators; b) The indicator targets; and
www.integrityglobal.com 19
c) The results achieved to date and time remaining.
Key findings • Lessons from partner-implemented community awareness raising activities has resulted in MyJustice adapting its approach to focus on critical partner capacity development to strengthen activity quality. • Justice centres are reaching more townships and people, and justice service providers are being trained, but user satisfaction levels and improved capacity of trainees is challenging to measure. • CBDR actors are being trained but knowledge and behaviour change will be challenging to evidence during the programme period. • Government buy-in to the MyJustice approach and research findings indicates progress in policy engagement and potential for influence.
4.1 Supporting communities to increase knowledge of their rights and pathways to justice MyJustice’s first result area focuses on the implementation of awareness raising interventions and support to CSO capacity development. The result area aims to increase people’s knowledge of their rights and justice pathways to enable them to demand fair and equitable justice services. MyJustice funds partners to deliver awareness raising activities at the community-level and provides capacity development support for partners to deliver more effective awareness raising interventions. MyJustice uses mass media for legal awareness raising, including the production of the television drama, The Sun, The Moon, and the Truth. Success within this result area is defined as: • People are reached through awareness raising activities. • Supported communities have increased knowledge of rights. • Partners increase their capacity to design and implement legal awareness activities.
Finding 16: MyJustice partners are reaching communities through awareness raising, but evidence of knowledge change is limited MyJustice awareness raising activities are being conducted by partners and through a multi-media campaign. Partner activities reached 12,078 people by the end of the first quarter of Year 3, thus already exceeding the programme target of 10,000 people. The programme has planned for additional awareness raising activities in Hpa-An, Mandalay, Taunggyi, and Tangoo. MyJustice has also commenced rolling out a coordinated multi-tiered, multi-media access to justice campaign for legal and rights awareness raising which aims to reach 2 million people. This includes support to the television drama series, The Sun, the Moon and the Truth, and planned development of public service announcements (PSAs), radio shows, and social media campaigns. MyJustice has engaged a consultant to support with designing monitoring and data collection mechanisms for this intervention. Given the early stages of this activity, there is limited evidence on the extent to which these media activities are reaching the poor and vulnerable, and the impact of these activities on the knowledge and perceptions of the target group. For the final evaluation, it will be important that this data is shared with the evaluator to assess effectiveness and impact of these activities.
www.integrityglobal.com 20
Some partners and service deliverers perceive to see indications of knowledge change among the targeted beneficiaries. Others emphasised it is too early in the process. One key issue limits the measurement of progress against this indicator: i) Partner monitoring of knowledge change is limited. Some partners conduct pre- and post- tests for awareness raising activities, but these tend to focus on how people ‘liked’ the activity and do not track substantive knowledge change.
• Recommendation 14: Prioritise target setting for Indicator 1.2 and communicate this with partners as soon as possible. When setting the target consider the ambition of knowledge change, given the context, quality issues with the initial interventions, and the remaining programme. • Recommendation 15: Utilise tools such as sampled FGDs or community surveys conducted several months following awareness raising activities, to track the impact of these activities on knowledge change.
Finding 17: Capacity development support to partners is a key strategy for achieving awareness raising objectives but the absence of indicators and monitoring tools limits the ability of the programme to measure effectiveness of this strategy There are both external and internal challenges to implementing awareness raising. While results are being achieved, CSO partners identified some external challenges to implementation, including: • Township and ward administrators are important gateways to the communities and can also be a barrier to implementing activities on time and with the appropriate reach. • Cultural norms and stigmas can limit the number of people who are receptive to awareness raising activities. • Staff safety and security concerns for some organisations.
In addition to these external challenges, MyJustice staff and partners have identified internal implementation (…) the approaches used for legal challenges. Through partner engagement, reporting, and awareness“ continue to be mainly the MyJustice CSO capacity assessment conducted in traditional with the focus being more on the number of people reached rather mid-2017, MyJustice and partners identified technical than targeting to support communities knowledge and skills gaps that may affect the quality of to take action relating to their justice awareness raising interventions and their ability to rights. contribute to increased knowledge of rights (Indicator MyJustice” Capacity Development 1.2). Strategy for Partners, p. ii In response to identified partner capacity issues, MyJustice made a strategic decision to incorporate capacity development of CSO partners as a specific workstream under Result Area 1. The programme determines that this workstream is important for increasing the likelihood of achieving the programme’s objective (that awareness raising interventions are well designed, appropriately targeted, and can lead to increased understanding and knowledge among the audience). To date, there is little evidence of MyJustice’s capacity development support leading to increased capacity of partners to design and implement interventions. This is due in large part to the timing of this support. The CSO strategy was completed in mid-2017 and new partners have only recently been engaged to deliver capacity training and mentoring support. While the CSO strategy offers suggested indicators and sources for assessing improved capacity, the MEL framework has not yet
www.integrityglobal.com 21
incorporated these. For instance, the MEL framework baseline is broad, and it is unclear how the endline should be calculated. The review team understands that partners delivering the capacity development activities will be monitoring progress and assessing the extent to which capacity has been developed. How MyJustice will use this partner data to report against results is unclear. In addition, some partners benefiting from capacity development activities have completed or are nearly completed their awareness raising activities under their current MyJustice grants. It is not clear if and how the programme will continue to engage these partners in these activities for the remainder of Year 3 and Year 4. • Recommendation 16: Insert the CSO capacity development baseline and targets into the MEL framework to track progress and assess the result of these activities. • Recommendation 17: Develop an indicator/s to monitor quality of interventions to assess the effectiveness of capacity development of partners, but also to quality control interventions. • Recommendation 18: Develop and apply a clear monitoring framework to assess CSO capacity development and supplement this with interviews and feedback sessions with partners to get their perspectives and inputs on the monitoring methods and tools. • Recommendation 19: Explore funding opportunities for partners that have participated in capacity development activities to continue to apply their new skills in awareness raising activities in target locations.
4.2 Increasing the availability and quality of justice services MyJustice’s second result area focuses on the availability and quality of justice services in the programme’s target communities, and behaviour change of duty bearers (those who are mandated to provide access to justice, such as judges). This result area includes five workstreams: 1. Increasing number of communities served by justice centres 2. Strengthening the quality of legal representation and paralegal services 3. Clinical legal education programmes 4. Mobile legal service directory 5. English language training to the UAGO
For the purposes of the review, we primarily focused on MyJustice’s engagement with the two ‘legacy’ justice centres in Yangon and Mawlamyine, and the engagement of International Bridges to Justice (IBJ), an INGO contracted to establish and manage four new justice centres in Taunggyi, Mandalay, Taungoo, and Hpa-An. (See case study of the Yangon and Taunggyi justice centres). Workstreams under Result Area 2 are delivered by a range of partners, some of whom are also engaged in awareness raising activities and training of paralegals and other justice actors. The workstreams are expansive but success is focused on key results principally related to: • Increased townships with complementary justice services • More people accessing justice services • Trained service providers • Improved capacity of service providers • Increased satisfaction of justice service users • Referrals from duty bearers to justice services
www.integrityglobal.com 22
Finding 18: Justice centres are reaching more townships and people, but measuring user satisfaction is challenging Through the establishment of four new justice centres, MyJustice has increased the number of townships in which justice services are delivered to 44 (of a target of 50) townships. The number of people accessing MyJustice supported justice services is close to 36% of the programme’s intended target of 12,500 people. As the new justice centres become more established, we expect these targets to be reached by the end of the programme period. Indicator 2.1 reads: 'Increase in number of townships with complementary justice services supported by MyJustice’. However, the concept of complementary justice services is not defined in the MEL framework and is not a focus of the indicator measurement. According to MyJustice staff, they originally planned to co-locate justice centres with other programme-supported services to create what they termed a small ‘eco-system.’ Given the focus of justice centres on criminal defence, and the array of other justice needs within communities, these other services would enable case referrals and tracking of beneficiaries through a range of justice pathways. Due to the programme decision not to undertake a second grants round, this is not the case with fewer complementary justice services in the target communities of the new justice centres. The programme seeks to measure increased satisfaction among users of MyJustice-supported justice services (Indicator 2.5). The Yangon Justice Centre (YJC) collects this data through post- case surveys conducted by centre staff. The YJC data indicates a 90% satisfaction rate among their clients through this method. MyJustice staff noted that it is challenging for them to undertake spot checks of user satisfaction as they rely on YJC to approach clients and because clients frequently move and are no longer contactable. According to the Taunggyi Justice Centre (TJC) staff, their lawyers implement pre- and post-case surveys with clients. MyJustice staff noted some concerns around the robustness of this data given the questionnaires will be administered by the lawyers themselves rather than a third-party. In addition, the TJC staff noted that in some cases per-tests are completed by family members of detainees, not the client themselves, and post- surveys have been challenging to complete when the client is imprisoned or is not in contact with the TJC after their release. MyJustice has not yet received the first set of user satisfaction data reporting from the TJC. • Recommendation 20: Review the expected change under Indicator 2.1 to assess if achievement is still possible given the limited complementary services. Revise the logframe if required. • Recommendation 21: Consider revising Indicator 2.5 to strengthen its accuracy. For example, a revised indicator that can also be a proxy for measuring ‘trust,’ could be the number of referrals by former clients.
Finding 19: MyJustice has trained justice service providers, but evidence of their improved capacity remains limited MyJustice supports partner training of justice service providers, specifically paralegals and lawyers to increase their capacity to deliver legal aid services to the poor and vulnerable. This support is captured in partner reporting and the programme’s information management database. To date, MyJustice has trained 255 service providers out of a target of 300. The perspectives of partners and the programme is that it is still too early in the process to see substantial change in the capacity of service providers to apply newly attained knowledge and techniques. Based on our assessment of the available data, this perception is valid. Capacity development is ongoing and the contextual challenges to apply new ways of working remain considerable (see Impact section for further analysis). There are several important issues that
www.integrityglobal.com 23
could limit the effect of capacity development on service providers within the timeframe of this programme. These include: • Paralegal training quality: The MyJustice capacity assessment identified quality issues with paralegal Language challenges: training provided by partner CSOs. The programme … I think that we missed a lot of has now contracted Namati, an INGO, to deliver “learning during translation process. training and capacity development support to We prefer a local professional trainer partners to better implement paralegal training. (maybe conducting the training himself or jointly with the foreigner • Lawyer training quality: MyJustice and IBJ are trainer) to have a better reviewing the programme’s lawyer training approach understanding when conducted in based on feedback from initial IBJ-implemented Myanmar language. We are also training that raised concerns regarding its contextual less shy/hesitant and/or more daring relevancy. to ask questions and more actively participate in discussions. • Language: Language barriers have been identified ” as limiting knowledge and skills transfer in trainings Paralegal and programme mentoring delivered by international, non-Myanmar speaking experts.
The programme is establishing ‘communities of practice’ for lawyers in Year 3. These communities of practice intend to be catalysts for capacity development through networking, knowledge sharing, and lesson learning. Feedback from attendees to other networking forums held between MyJustice-supported service providers and other justice Some new concepts and learning sector actors has been positive. One MyJustice-supported “are useful. I acknowledge many best lawyer noted that, ‘We get to know each other, we can practices – we just need some flexibility share our difficulties and needs. It allows us to build while trying to completely follow. [For example] we are forced to meet the relationships. I think these have led to change’. defendant in the police jail. Before this, It is not clear how the programme defines and measures maybe we tried once, and gave up when the first attempt failed. But now ‘demonstrated increased capacity’ of lawyers to deliver we are pushed to try multiple times and legal aid (Indicator 2.4). Some data on lawyers applying we can meet the accused in jail in new practices, such as making bail applications, exists. At some cases. It is good practice. We the legacy justice centres, there is possibly more data see some positive results. available that can be reviewed as a proxy for improved Lawyer, Justice” Centre capacity. There is also some anecdotal evidence of changed knowledge and practice among lawyers however, the extent is unknown given the absence of systematic monitoring. Without a clear definition of increased capacity, and in the absence of a baseline or target for Indicator 2.4, the programme will be unable to track and eventually report achievements under this result area. • Recommendation 22: Clarify how the indicator 2.4 is to be assessed and establish a baseline and target to enable monitoring and evaluation.
Finding 20: English language training is achieving results but its contribution to improving the quality and accessibility of justice services at the community-level is tenuous There are indications that the English language training provided by MyJustice is a useful tool in strengthening a positive and communicative relationship between the programme and the UAGO, a key rule of law institution in Myanmar. Indications include: • Positive feedback from senior UAGO figures.
www.integrityglobal.com 24
• A steady increase in the number of classes requested and offered since the start of the programme, including a specifically requested class for directors. • Staff are authorised to attend classes held during office hours.
The relationship between the programme and UAGO is considered vital for buy-in and for progress in MyJustice’s policy engagement work under Result Area 4 (see 4.4 for further examination). However, the link between English training and Result Area 2 objectives of improving justice services is tenuous. The programme needs to reconsider where this activity best sits in its programme logic and change pathways. • Recommendation 23: Align the English language training activity to Result Area 4 to more accurately reflect the linkage between the activity results and the intended change.
Case Study: Yangon and Taunggyi Justice Centres The Taunggyi Justice Centre (TJC) and Yangon Justice Centre (YJC) are two of six MyJustice supported-justice centres in Myanmar. YJC is one of two ‘legacy centres’ established under the UK DFID-funded Pyoe Pin programme and integrated into MyJustice in August 2015. Programme support to the YJC is well-aligned to MyJustice’s objectives, but the fact that it was an inherited project created some challenges in obtaining buy-in for the programme’s learning-focused approach. The programme also faced early difficulties in measuring the quality of the centre’s services and helping to build its organisational capacity. MyJustice embarked on the design of the new justice centres, including the TJC, with a goal to learn and adapt from the lessons of the legacy centres.17 A sample of the programme’s learnings and responses are presented in Table 4. The substantive focus, service structure, and overall objectives of the YJC and TJC do not fundamentally differ. Both seek to improve access to free legal advice and representation for the indigent criminally accused. However, MyJustice’s decision (based largely on staffing resource and capacity considerations) to out-source implementation of the new justice centres to a partner INGO, IBJ, has also had challenges. The partner brings experience of similar work in a range of contexts outside of Myanmar, but local stakeholders have flagged concerns, for example, regarding the relevance of its implementation approach to the specific Myanmar context.
Table 4: Select examples of MyJustice’s learning
Lessons from legacy centres Response in design of new centres
Staffing structure needs to be New model based on an initial small staff structure, and an financially sustainable. affiliated network of pro bono lawyers. Organisational capacity development New model involves IBJ staff oversight and direct support to is most effective when considered from develop operating systems and build organisational capacity. the outset.
Strategies are needed to address MyJustice undertook comprehensive scoping to identify new resistance from stakeholders such as justice centre sites, including sensitisation of key local government government actors, local communities, and community actors. and private lawyers. MyJustice commissioned rapid context briefs to identify localised
17 MyJustice’s lessons were informed by an external evaluation of the impact and progress made by the YJC and the Mawlamyaing Justice Centre from establishment to June 2016. See Debrah Mercurio, Legal Aid in Myanmar: The Yangon and Mawlamyaing Justice Centres, Impact and Progress Report, (June 2016).
www.integrityglobal.com 25
context dynamics and key stakeholders
Given the differing contexts in which they were both established, and the fact that TJC is only recently operational, it is not possible to directly compare the operations and resulting effectiveness and impact of two centres. However, both centres do provide important learning for informing national policy discussions on legal aid; informing a potential second iteration of the programme and expansion of the justice centre model; and for other local and international development actors supporting legal aid services in the country. Specifically, evidence suggests that an effective justice centre model should be based on: • a strong political economy engagement approach including a deep understanding of the context dynamics within target communities and the justice sector itself; • the flexibility to be able to respond to context dynamics; and • the ability to address priority justice needs within target communities.
Context considerations The YJC was the first justice centre of its kind in Myanmar, established at a time when the concept of legal aid was virtually unknown amongst government, lawyers, or communities. By the time the TJC was operational, the Legal Aid Law (2016) and a government-formed oversight body were in place. Yet understandings of legal aid and legal aid service provision remains limited within communities and The YJC model government, with consequences on the acceptance and (…) was about demonstrating what support of the justice centre work. In Taunggyi, “was possible … let’s make it happen. stakeholders noted that justice centres are perceived by ” YJC stakeholder many as contributing to a ‘disruption of the market,’ i.e. taking paying clients away from private lawyers who are also struggling to earn a decent income. One respondent noted that in Myanmar even the very poor will find a way to pay for legal services when they are in desperate need of help. Both the YJC and TJC refer cases to private lawyers. The peer network and the reputation of the YJC head were critical contributing factors to the YJC’s acceptance within the profession during its establishment and beyond. The TJC model also prioritises developing a strong peer network through the provision of training and establishment of a community of practice. Currently referrals are only made to a group of about 20 lawyers that are personally connected to the senior criminal advisor. Some respondents felt that the TJC needs to make greater effort to garner support from the broader legal profession in Taunggyi to facilitate acceptance of the TJC operations. The TJC staff noted that the allowance provided to lawyers taking referrals is very small and the internal claims system is very burdensome. In reality, most of the lawyers do not request payment of the allowance. In the long term this may tarnish the reputation of the TJC amongst legal professionals. It could also negatively impact IBJ’s reporting of case numbers handled by TJC and its network, including by affecting value for money calculations and analysis of priority justice needs in the community. • Recommendation 24: Review the TJC referral system with TJC staff and lawyers to address barriers to payment.
Addressing specific justice needs
www.integrityglobal.com 26
Indigent criminal accused are a vulnerable group according to the programme’s definition, and respondents in Taunggyi and Yangon did not dispute their need for free legal representation. However, research from MyJustice and other organisations emphasises that there are a range of other civil and administrative justice problems that people experience in their daily lives. Yet there are very few justice services available to address these needs. Yangon respondents felt that the YJC was generally able to respond to broader justice needs because it had established the University Legal Clinic, which adopts civil cases. YJC lawyers also provide legal advice (but not representation) on a range of issues beyond criminal defence. By contrast, respondents’ dominant concern in Taunggyi is that the new centre only provides legal representation to indigent criminal accused. The limited number of other Taunggyi service providers further compounds this concern. In addition, some community members reportedly have a negative perception of the TJC, which is seen to only support ‘criminals’ (and not victims of crime). TJC awareness-raising activities attempt to mitigate this negative perception. So far, the TJC has been able to refer non-criminal cases and requests for assistance by victims, such as those experiencing domestic violence, to local CSOs or private lawyers. Neither justice centre has specific targets for the number of female clients or ethnic or religious minorities receiving legal assistance. Gender and minority data is however disaggregated in both case database systems and reported to MyJustice. 31% of the 917 YJC clients within the current project period are female. It was not clear from discussions with YJC staff as to the extent to which the YJC prioritises assistance to specific vulnerable groups. In fact, the approach appears to be to take as many cases as possible, including those who do not technically fulfil the core income criteria but have other considerations such as supporting large families. However, for women, children and the disabled, YJC staff stated they apply a more relaxed approach to the income criteria for assistance. IBJ also noted that they loosen the financial guidelines for linguistic, ethnic or religious minorities, although the TJC staff themselves stated there were no special rules for specific vulnerable groups. • Recommendation 25: Review the legal aid criteria, existing case data, and relevant baseline data with each justice centre to identify how best the centres can: o respond to the priority justice needs of their communities; and o prioritise legal representation for the most vulnerable.
Conclusion The justice centres, both legacy and new, respond to a clear access to justice need within their target communities. Lessons from implementation show that a context-driven approach can facilitate acceptance and value for the justice centre concept by the community and wider legal profession. As a result, large numbers of vulnerable individuals can receive necessary legal assistance. MyJustice is committed to being people-centred, locally-led and evidence-driven. The programme’s learning from the centres and about the broader justice context in Myanmar should continue to inform implementation and adaptation of the justice centre model. This will ensure that the model reflects the programme’s guiding principles, responds to the localised justice needs of the target communities, and effectively contributes to increasing access to justice in Myanmar.
www.integrityglobal.com 27
4.3 Strengthening Community Based Dispute Resolution The third focus area for the MyJustice programme is to strengthen community based dispute resolution (CBDR) mechanisms. Mercy Corps and Norwegian Refugee Council are the main MyJustice implementing partners although other partners, such as Terre des Hommes, also contribute to this result area. Activities are delivered across five workstreams: i) Capacity development of Ward/Village Tract Administrators (W/VTAs) on dispute resolution skills; ii) Training GAD Deputy Staff Officers (township level), on dispute resolution skills; iii) Capacity development of community leaders on land and rights-based dispute resolution; iv) Engaging with communities on land and rights-based dispute resolution; and v) Increasing knowledge base on dispute resolution practices.
MyJustice’s success in this result area focuses on the following areas: • Number of CBDR actors trained. • Increased knowledge and understanding of fair approaches to CBDR by trained actors. • Number of CBDR actors who use fair approaches for dispute resolution. • Increased number of CBDR users who report a fair approach being used.
Finding 21: CBDR actors trained using adapted methodologies but knowledge and behaviour change will be challenging to evidence during the programme period Mercy Corps and the Norwegian Refugee Council both provide dispute resolution skills training to WVTAs. MyJustice collaborated with Mercy Corps on their training methodology to ensure it encompassed a rights-based focus. Both Mercy Corps and MyJustice staff gave positive feedback on this collaborative process and the result. Trainers reportedly received some critical feedback on the Mercy Corps training that it is quite theory-based and hard to apply in practice. However, the review did not include interviews with trainee WVTAs (due to the WVTA elections being held at the time of the review) and so could not cross-check this feedback. The methodology will be further tested in the next round of trainings. MyJustice partnered with Mercy Corps and UNDP to design and deliver tailored training to GAD administrators and staff officers as part of their annual training curriculum delivered by the Institute for Development Administration (IDA). Mercy Corps perceives engagement with the GAD to be particularly strategic given the hierarchical local administration structure. Feedback from the IDA regarding the training was relatively positive, although respondents noted that the facilitation style of the training was different to that of the IDA training. This has created expectations amongst trainees that all IDA trainings will be facilitated in this way moving forward. While this is an indication that the training style was well received by the trainees, this is an area of sensitivity for the IDA management and requires further engagement by Mercy Corps and/or MyJustice. The programme reported that the unexpected announcement of nationwide local WVTA elections in December 2017 resulted in a suspension of CBDR training activities until election results are confirmed and Mercy Corps has been able to review its participant selection to ensure training targets newly elected and re-elected WVTAs. As a result, there is currently limited evidence of WVTA knowledge change or the application of new practices. Mercy Corps has developed a tool to track whether fairer approaches to CBDR are used in targeted communities. This tool will need to
www.integrityglobal.com 28
be tested once the training recommences. Considering the unexpected delays with this activity, knowledge and behaviour change is not expected within this programming period.
4.4 Using evidence and knowledge sharing to inform justice policy development The fourth area of focus for the MyJustice programme is to use learnings from the other result areas to strengthen and inform justice policy development and implementation by government actors. Within Result Area 4, MyJustice activities are delivered across five workstreams, namely: i) Improving capacity of MJ partners to engage with policy makers; ii) Generating policy-oriented resources; iii) Fostering a community of practice; iv) Technical and policy guidance to government on access to justice policies; and v) Supporting the Union-level JSCB and select state/regional JSCBs.
Key successes under this result area are: • Research and analysis products are developed to inform the programme’s policy audiences. • Partners demonstrate increased capacity for policy engagement. • MyJustice undertakes strategic engagements to inform access to justice policy development and implementation. • MyJustice knowledge sharing events and briefings use programme generated evidence.
Finding 22: Government buy-in for the MyJustice approach and research findings demonstrates progress in policy engagement and potential for influence In Year 2, major programme research products were developed that facilitated an increase in MyJustice’s policy engagement work. The programme has used these key research products as an entry point for engagement with critical government stakeholders. External stakeholder feedback on the research content and the quality of the presentation has been positive. One stakeholder noted that the programme’s short and concise policy briefs have been particularly well-received. One of the main research products produced by the programme is the results of the nationwide KAP survey. MyJustice has presented the KAP findings to a range of government stakeholders. Representatives from both UAGO and the OSCU provided positive feedback, and indicated that they will The research they are doing is use this data to inform their own internal strategies. “effective for us. For a project to These government stakeholders also indicated an succeed, it depends mainly on the appetite to further expand the research, noting they research. Based on the research, we can get useful knowledge. All the facts would also be interested in learning more from people at and figure obtained from them can be the community level that are using the justice services. used by us. This feedback suggests government stakeholders both ” Government Stakeholder support and value the programme’s community-led approach. A key result reported at the end of Year 2 was MyJustice’s appointment, along with UNDP and Robert Pe (legal advisor to Aung San Suu Kyi), as a member of an Advisory Group to support the Union-level Joint Co-ordination Body for Rule of Law Centres and Justice Sector Affairs (also referred to as the Justice Sector Coordination Body). This appointment was an unexpected result attributed both to the programme’s successful engagement with the UAGO and the Team Leader’s
www.integrityglobal.com 29
strong relationship with the Attorney General. External stakeholders regarded the appointment of international advisers to a government body as unprecedented. MyJustice also supports the state/regional JSCBs in Mon State, Kayin State, Yangon Region and Bago Region. MyJustice provided a presentation of the KAP findings to the Yangon Region body. Feedback from Yangon body members, who identified MyJustice as their key international partner, was generally positive. Based on this evidence, MyJustice is on track to continue using evidence to engage with policy makers. The programme is committed to supporting partners to engage with policy makers. Indicator 4.2 of the programme’s logframe seeks to measure increased capacity of partners for policy engagement. To date, engagement with government has been done directly by the MyJustice team. However, plans are underway to support greater partner engagement. MyJustice has developed a Policy Brief Toolkit to impart practical tips to partners and other CSOs to ‘write engaging, targeted and evidence-based policy briefs based on their own programme learning.’ The CSO strategy also includes policy engagement capacity building activities. Several respondents flagged that an added value of MyJustice engagement with the JSCBs is that they enable the building of connections or bridges between these government bodies and local CSOs. Given the nascent state of these partner engagement activities, there was no evidence currently available to assess partners’ increased capacity or how this contributes to achieving programme results. MyJustice should consider these monitoring issues when addressing the recommendations in section 4.1 above.
5 Impact Overview and Key Findings This section examines the extent to which programme activities are beginning to enable greater access to more inclusive, accountable, and responsive justice services for people in target communities, and especially the poor and vulnerable. It explores the extent to which the expected changes can be evidenced, based on a review of current results, articulated change pathways and programme assumptions, identified challenges, and key external factors.
Key findings • While there has been progress within result areas, there is limited evidence of wider change to date. • Corruption and community distrust of the justice system remain key barriers to achieving the programme’s desired impact.
Finding 23: While there has been progress within result areas, there is limited evidence of wider change to date Increased awareness in communities: An important component of MyJustice’s intended change pathway to improving access to justice, entails creating more opportunities for community members to know their rights and options of pathways to accessing justice. The programme is not yet systematically collecting data on people’s knowledge change. In the absence of this data, the review examined the extent to which assumptions underpinning this change pathway hold true. To date, the quality of partner interventions has been mixed and a key assumption underpinning this change pathway has not held true – that CSOs have the capacity to deliver quality awareness raising activities. In response MyJustice has initiated CSO capacity development interventions. Time is now needed for both building CSO capacity and for partners to deliver strengthened
www.integrityglobal.com 30
activities. It is therefore unlikely the programme will be able to achieve the desired change in the remaining programme period. A re-examination of the change pathway may be required. • Recommendation 26: Begin systematically collecting monitoring data on knowledge change in target communities where interventions are already considered ‘good quality’ so as to assess the impact of these interventions on knowledge change.
Strengthened justice services: The programme aims to improve the capacity of justice service providers, such as lawyers and paralegals, to provide a higher standard of legal aid services. There is limited data on capacity change among service providers. However, key stakeholders agreed with the programme assumption that the capacity of service providers is low. MyJustice’s capacity development support to lawyers, is predicated on an assumption that lawyers are keen to learn and apply new practices and that lawyers (defence lawyers in particular) have a crucial role to play in ensuring access to justice and rule of law is a concept and value that can take hold. Interviewed stakeholders had varied perspectives on these assumptions. Some lawyers highlighted that senior lawyers are more resistant to change than junior lawyers for a range of highly-contextual reasons. One senior lawyer agreed that lawyers have a central role to play in promoting access to justice, but this depends on the younger generation of lawyers. However, junior lawyers face significant challenges in learning and applying new techniques due to the hierarchical nature of the legal profession. One key informant highlighted that the ability of lawyers to apply new techniques regularly is still ‘the exception.’ MyJustice is one of several programmes, including Promoting Rule of Law Project (PRLP) and UNDP, to provide capacity development support to lawyers. Some MyJustice-supported lawyers have attended trainings provided by these other programmes. MyJustice encourages engagement with other programmes to improve lawyers’ overall capacity. This is an important and useful strategy, considering the low capacity of service providers and limited resources of the programme. However, this also means that the direct contribution of the programme to developed capacity will be less clear. • Recommendation 27: Incorporate questions into Result Area 2 monitoring tools that ask lawyers about other training and support they receive from other programmes to capture potential overlaps and complementarity. • Recommendation 28: Clarify in the MEL framework what skills will be assessed to measure increased capacity of legal aid lawyers in order to measure progress against Indicator 2.4.
Increased demand for justice services: The programme’s theory of change expects that through knowledge generation people will be able to demand justice services. The programme assumes that by strengthening justice services to be more responsive to people’s needs and increasing their availability, more people will use them. There is limited evidence either of increased knowledge and awareness or more responsive service delivery at the present time. The lack of ‘ecosystems’ in target communities (see section 4.2) impacts the programme’s ability to track the ‘full circle’ of the programme’s theory of change – i.e. a person who receives knowledge, acts on that knowledge by seeking a justice pathway, and receives justice services of a higher standard. In some locations, such as Hlaing Thayar, tracking the circle might be possible due to the number of MyJustice-supported organisations and service providers, including the YJC satellite office, in that area. However, currently the programme does not undertake systematic tracking of beneficiaries. It therefore remains unclear whether this expected change is likely within the programme timeframe and whether the programme has the monitoring systems in place to measure this change.
www.integrityglobal.com 31
• Recommendation 29: Undertake an independent justice monitoring case study in Hlaing Thayar that tests the theory of change logic by tracking beneficiaries and focusing on the process - what role did the awareness raising have, how did the justice process work – as well as the result, i.e. was access to justice attained?
Changed behaviour of formal justice actors: The programme expects to change the behaviour of formal justice sector actors such as judges and police officers, by improving the quality of justice services being provided. The programme intends to improve the quality of justice services through capacity development support of service providers. To date, there are limited data points on behaviour change. The review did not include interviews with formal justice sector actors to test the programme assumption or degree of change. Partner reports indicate some evidence of behaviour change. Some interview respondents also provided anecdotal evidence of behaviour change, for example: • ‘In the past, they were not happy with us; but it has been changed. The judge [does] not favour other side although we do not bribe. Probably because of the justice centres’ back up’ – Lawyer • ‘In the past, some police don’t like paralegals doing random visiting to inmates kept in the jail at court. This is also changing too. They now let us talk with the inmates in the police and court jail (…) They cooperate with us.’ - Paralegal
One respondent challenged the expected change process by noting that judges will be influenced by their superiors. Therefore, if the Office of the Supreme Court shows acceptance and value for the work of legal aid lawyers, this will be most effective in changing judges’ attitudes and behaviour. MyJustice is encouraging partners to collect qualitative data on behaviour change of formal justice actors through case studies. Following the ROM review, MyJustice developed an indicator to track the percentage of clients referred by police and judges. According to reports, the percentage of referrals by these officials has increased in Yangon with some limited change observed in Mawlamyine. However, there is no baseline or target set for this indicator, which means it is not possible to track progress. The programme is also reassessing the reliability of this indicator after some indications that police in Yangon were extracting bribes from clients before referring them to the YJC. More widespread behaviour change - outside of the limited number of judges that are regularly engaging with MyJustice-supported lawyers - is not expected through this programme alone. Contextual factors, such as the Legal Aid Law (2016), could contribute to or limit this broader change. • Recommendation 30: Consider whether the programme’s engagement with the OSCU on the Legal Aid Law can be leveraged to promote behaviour change of judges and, if so, articulate this in the result area strategic framework and allocate necessary resources.
Fairer community based dispute resolution mechanisms: The programme expects that fair community based dispute resolution mechanisms will improve access to justice for vulnerable people. This assumption remains untested and requires further evidence from the Mercy Corps and Norwegian Refugee Council activities. The assumption is that vulnerable groups can and desire to access these mechanisms. Typically, dispute resolution will take place in an open office, which can discourage individuals from using these mechanisms. One stakeholder noted that women, for example, are resistant to using such mechanisms due to their public nature.
www.integrityglobal.com 32
• Recommendation 31: Engage with partners to further test the assumption underpinning CBDR engagement and identify potential barriers to access that can be addressed through partner training and mentorship activities.
Justice policy change: MyJustice has had notable results in their policy engagement component with increased opportunities to engage with policy makers and influence policy-making. However, the programme does not capture, through its MEL framework, evidence of changed policies as a result of MyJustice involvement. Further, given the slow pace of reform generally in Myanmar, the impact of MyJustice’s work may not be visible during the life of the programme. The programme’s contribution to longer-term change is predicated on an assumption that there are feasible entry points and opportunities to affect policy development. Evidence of these entry points and opportunities include MyJustice’s advisory role to the Union-level JSCB and the programme’s involvement in the recent national justice conference. Within the programme timeframe, change is expected in terms of how partners engage with policymakers and how research and evidence is used to inform policy development. There is an implicit assumption that government and other key stakeholders will be receptive to research and evidence generated through MyJustice. There is already evidence that this is the case, with key stakeholders from the UAGO and OSCU engaging positively with MyJustice’s most recent research findings. • Recommendation 32: Establish a system for monitoring how MyJustice research is used and applied by policymakers to their daily work. Such data would usefully inform future research outputs, dissemination strategies and programme follow-up.
Finding 24: Corruption and community distrust of the justice system are critical contextual barriers to programme impact MyJustice research identifies corruption and high public distrust in the formal justice system as important barriers to access to Corruption is [the] main justice. The baseline report emphasises that while limited “challenge. They will win if they knowledge is one factor contributing to limited demand by can bribe the judge. ” communities for justice services, the prohibitive cost of justice Lawyer, Justice Centre services and the widespread distrust of the justice system are important additional factors. Corruption: MyJustice staff and partners acknowledge that corruption is rife within the justice system and bribes are a regular part of legal proceedings. Lawyers noted that sometimes clients will prefer to drop their legal representative and pay a bribe – or confess – to more speedily conclude their case. Others noted that refusing to pay a bribe can negatively impact the lawyers’ reputation and perception of their ability to do their job. At the same time, YJC lawyers suggested that their refusal to pay bribes, in accordance with MyJustice’s zero-corruption policy, is being accepted by some justice sector stakeholders and may be appreciated by some clients. The programme has not established or defined an expected change regarding corruption within the justice system. • Recommendation 33: Engage with the justice centres and other service providers to capture learning on how corruption is affecting their ability to deliver quality services and jointly identify processes to overcome these challenges and/or establish realistic indicators of success.
Distrust of the justice system: The capacity development of lawyers might partially address issues of community distrust in the justice system. However, programme activities do not target specific justice actors, such as the police, whose actions directly impact people’s perceptions about
www.integrityglobal.com 33
the justice system. Because the programme is predominantly working at the community-level, its wider impact on the justice system as a whole will be limited by other underperforming segments, such as law enforcement, the judiciary and corrections (see section 7 Coherence for further discussions and recommendations on this point). 6 Sustainability Overview and Key Findings This section examines the extent to which programme activities, particularly those implemented by partners, are likely to be sustained when donor funding is no longer available. The review focused on identifying early signs of sustainability, such as evidence that MyJustice is building partner
Key findings • Future programme funding has yet to be secured and sustainability contingency planning is important given recent global political developments. • Lesson learning and programme advocacy to government bodies are important strategies for promoting long-term sustainability of justice centres. • CSO capacity development activities indirectly contribute to sustainability but organisational development needs remain. capacity to continue their work after the programme period or is developing sustainability plans for legal aid service provision with partners and policy makers.
6.1 Sustainability of the programme Finding 25: MyJustice sustainability is reliant on continued EU funding and key staff personalities MyJustice and the EU both indicated that EU-funding of a second iteration of MyJustice was a strong possibility. No alternative funding sources for the programme were immediately apparent. British Council and EU representatives were uncertain if or how Brexit18 would impact the eligibility of the British Council to receive EU funding in the future. However, several respondents did not perceive it to be a major obstacle to the ability of the British to implement a second iteration of the programme. This was because of the United Kingdom’s commitment to continue its contributions to the current EU Development Fund which goes through until 2020. Lessons from the programme (and international practice generally) show that deep-rooted change within the Myanmar justice system will take much longer than four years. The programme is establishing a critical foundation for change through its relationship building and capacity building activities, and programme learning from an array of pilot activities and tested approaches. However, more time is needed to enable this change, through continued delivery, learning and adaptation. The absence of committed donor funding to extend the timeline of the programme is a key challenge in enabling sustainable change. Several stakeholders perceived the success and reputation of the programme to be tied to specific personalities within the MyJustice team, including the Team Leader. Interviews with other key government stakeholders and partners validated this perception. Other respondents emphasised the importance of a secession plan in the event key personalities were not involved in a second iteration of the programme, and suggested empowering middle management to represent and
18 Brexit is the term for the planned departure of the United Kingdom from the EU..
www.integrityglobal.com 34
promote the programme. One respondent reflected that the fluid political environment meant government appointments will likely shift, which could impact the programme and its level of access to government representatives regardless of these key personalities. MyJustice is aware of these factors which challenge programme sustainability. MyJustice is already engaging with the EU and British Council in discussions regarding Brexit and programme sustainability. The Team Leader is also creating opportunities for more staff to engage with policy makers and build programme relationships. These are important steps to address key challenges to sustainability. The impact of these efforts on programme sustainability can be further assessed at the final evaluation stage. • Recommendation 34: Continue to engage the EU and British Council in discussions regarding programme sustainability as a matter of urgency, including contingency planning in case future EU funding is unavailable. • Recommendation 35: Continue to empower staff at all levels, and particularly those likely to participate in a second iteration of the programme, to more actively engage in programme and activity-level decision making and representation with government officials.
6.2 Sustainability of MyJustice partners The programme is taking some steps to promote partner (including justice service providers) sustainability through, for example: • Policy engagement work regarding the national legal aid system; • Technical capacity development support, including CSO capacity development and training of CBDR actors; • Limited organisational development support to some partners and service providers; and • The promotion of a pro bono culture within the legal profession.
However, MyJustice has not begun explicit sustainability planning with individual partners. It is unlikely that partners will be able to continue their activities beyond the programme period without further donor funding.
Finding 26: MyJustice supports justice centres through policy advocacy and organisational strengthening but centres will require donor funding to continue operating beyond the programme period Myanmar currently does not have a fully operational national legal aid system.19 MyJustice is providing direct technical advice to policy makers, including the ULB and the state/regional JSCBs, regarding the design and implementation of a legal aid system. It also promotes engagement between these policy makers and the MyJustice justice centres with the aim of encouraging the integration of the justice centres into a future system. For example, senior government officials were invited to attend the openings of the new justice centres. These programme efforts are positive steps towards supporting the sustainability of the centres. Simultaneously, the programme seeks to build the organisational capacity of the justice centres to aid sustainability. Efforts by MyJustice to promote strategic planning and management reform
19 Despite the passing of the Legal Aid Law in January 2016, the ULAB, which will determine and implement the government’s legal aid policy, has only just been established. A national legal aid system will require many years and continued donor support to become fully functional.
www.integrityglobal.com 35
processes within the legacy justice centres20 have had limited impact to date. The programme embedded international volunteer lawyers at the centres to support strategy planning. However, justice centre management resisted engaging substantively with these volunteers. This was an important lesson for the programme, and one that informed a change in approach. In Year 3, MyJustice staff are now directly supporting organisational development of the legacy justice centres, including the strengthening of case management data systems.21 MyJustice and its partner, IBJ, designed the new justice centres22 with sustainable cost considerations in mind, including adopting a model that relies on a small number of core staff and a network of pro bono lawyers. The partnership agreement also requires IBJ to provide organisational capacity development support for centre staff to enable the centres to eventually operate as local CSOs. To date, there is little evidence of a focus on the development of operational and management skills, with IBJ focusing on enhancing lawyer capacity in the first instance.23 The current trajectory indicates that these new centres will not be able to sustain their activities beyond the MyJustice programme period without external funding support. • Recommendation 36: Prioritise support to legacy justice centres to develop funding strategies for continued and diversified donor support beyond the programme period, including for example, proposal writing and donor relations skills development.
Finding 27: Programme efforts to promote a sustainable pro bono culture within the legal profession are nascent but lawyers appear willing to collectively advocate for access to justice The Results-Orientated Monitoring (ROM) report recommended that MyJustice further mainstreams the culture of pro-bono work among its partners to support sustainability. There is a strong history of pro bono work in Myanmar however this has been negatively impacted by political, professional, financial and other challenges over the years. Sustainability of legal aid services requires that pro bono practice be rooted in a sense of professional obligation and duty to protect fundamental rights (not merely charitable inclinations). Several respondents emphasised that this pro bono spirit requires lawyers to work together: “Lawyer unity is really important. It’s the only way things will change.” (External stakeholder). This perception gives support to the relevance of the programme’s recent efforts to design activities that encourage a collective commitment to promoting access to justice. Activities include the establishment of communities of practice; and an externship project for law students to work at the new justice centres. These activities had not commenced at the time of the review and therefore their impact on sustainability could not be assessed. However, given the importance key stakeholders gave to collective action among lawyers, these activities are a positive step in the programme’s contribution to enabling sustainable change.
20 Established under a DFID funded project in 2013 in Mawlamyine and Yangon. 21 The programme assumes that efficient case management systems will support organisational efforts to track cases and improve value for money. The data collected in these systems also has the potential to position justice centres as a key source of reliable data to inform legal aid policy. 22 Established in Mandalay, Taunggyi, Taungoo and Bago. 23 Interviews with TJC staff revealed a tension between staff needs and the IBJ management team responses. TJC staff have limited daily interaction with IBJ staff. Communication tends to be via email and in English which can be challenging for TJC staff and leads to misunderstandings. TJC staff desire greater one- on-one engagement with IBJ staff to address a range of concerns including the efficiency and accuracy of case management systems, and the practicality of some Monitoring and Evaluation tools.
www.integrityglobal.com 36
Finding 28: MyJustice’s CSO capacity development support indirectly contributes to sustainability but does not address critical organisational development needs that more directly promote partner sustainability Sustainability discussions between MyJustice and partners around future funding strategies are nascent. Putting in place organisational Securing funding is the major sustainability concern for “structures and ensuring the national partners. Several partners noted that gaps in funding, colleagues have the skill to manage the even of only a few weeks, can be highly detrimental to programme at an international level to the ability of their organisations to continue operations. be competitive for international funding without international implementers is Some partners believe, given the limited number of local key. CSOs undertaking access to justice work in Myanmar, ” that funding for their activities will be relatively easy to External stakeholder secure from other donors. The review revealed there is no uniform approach to funding strategies amongst partners. However, the majority appear to have a project-focused approach (i.e. seeking funding on an activity or project basis), rather than an approach focused on institutional sustainability. Partner respondents identified a range of organisational capacity needs and a desire for more tailored mentoring support to strengthen organisational capacity. MyJustice’s CSO capacity development work currently provides technical knowledge and skills capacity support to 15 local partners. MyJustice elected not to support other identified organisational needs due to the programme’s time, budget and staffing limitations. MyJustice capacity development support coupled with a commitment of partners to find ways of continuing their work within a niche market will contribute to activity-based sustainability but will have limited impact on long-term organisational sustainability. • Recommendation 37: Engage with all partners to support the development of financial sustainability strategies. • Recommendation 38: Incorporate lessons learned regarding local CSO organisational capacity needs into planning for a second iteration of the programme.
7 Coherence and EU value added Overview and Key Findings This section considers the extent to which MyJustice aligns with EU policies and contributes to EU value add in Myanmar. It examines how the programme complements other justice sector
Key findings • MyJustice programme approach and objectives generally align with EU policies and priorities in Myanmar. • The MyJustice-EU partnership encourages two-way sharing of expertise and opportunities exist for greater alignment and coordination with other EU/member states programme. • High-level justice sector coordination by MyJustice and the EU encourages complementarity of interventions and opportunities exist for MyJustice to support stronger coordination at a community-level. programming and considers opportunities for strengthening this complementarity.
www.integrityglobal.com 37
7.1 Coherence with EU priorities Finding 29: MyJustice approach and objectives generally align with EU policies and priorities in Myanmar The EU Delegation to Myanmar was opened in September 2013 to support Myanmar’s democratic transition. The EU Multiannual Indicative Programme (2014-2020)24 articulates the EU's approach to its engagement in Myanmar. MyJustice’s approach and scope directly align to the EU’s objectives, pillars of engagement and key priorities, as outlined in Table 5.
Table 5: MyJustice alignment to EU priorities
EU Programme MyJustice Programme alignment
Overall strategic objective: To MyJustice intends to: ‘Improve access to justice and legal aid for support peace, security and the poor and vulnerable, develop legal capacity of justice sector stability in the country, while professionals and strengthen selected rule of law institutions to promoting inclusive growth and better fulfil their mandates.’ sustainable development, through the on-going reform process.
Specific Objective 2: ‘Legal and The programme contributes to the EU expected results under institutional capacity of justice Specific Objective 2: sector and law enforcement 2.1. Improved administration of justice, efficacy of law agencies is strengthened and enforcement agencies and level of independence from political access to independent, impartial and other forms of influence and transparent justice and legal 2.2. Enhanced legal knowledge, skills and institutional capacity of aid is improved.’ the judiciary 2.3. Increased access to justice, especially for vulnerable groups Pillars of engagement: The programme responds to Pillar 3, Specific Objective 2, namely: I. Rural development / ‘Legal and institutional capacity of justice sector and law Agriculture / Food and enforcement agencies is strengthened and access to independent, nutrition security impartial and transparent justice and legal aid is improved.’ II. Education III. Governance / Rule of law / State capacity building IV. Peacebuilding support
24 European Union Development Cooperation Instrument: Multiannual Indicative Programme (2014-2020) Myanmar/Burma, at https://eeas.europa.eu/delegations/myanmar- burma_en/19059/EU%20Multiannual%20Indicative%20Programme%202014-2020.
www.integrityglobal.com 38
Priorities: MyJustice prioritises working with communities and civil society (often referred to as a 'bottom-up' approach). The programme also • Strengthening of a people- engages with government justice stakeholders and through its centred form of governance policy work seeks to be a bridge between community and • Improving public service government. delivery to meet people's needs The programme is an explicit access to justice programme that • Strengthening the involvement promotes greater understanding and respect for the rule of law and of the private sector and civil rights within communities and justice institutions. society • Strengthening rule of law and The programme undertakes some activities in mixed control areas respect for and promotion of (such as in Kayin State), and other activities intend to examine human rights issues of ethnic minorities and land rights and barriers to access to • Specific interest in ethnic and justice. border areas
While MyJustice staff and EU stakeholders agree there is general alignment between MyJustice and EU priorities, one area of potential tension is regarding the situation in Rakhine State.25 In January 2018, the EU announced a new €5 million programme to support Rohingya refugees from Rakhine in Bangladesh. The EU Delegation continues to explore how best to respond to the humanitarian crisis in Myanmar while at the same time being cognisant of the considerable political sensitivities around any engagement. The EU Delegation and MyJustice management staff have had a constructive dialogue regarding the potential role for MyJustice in Rakhine, given the justice concerns in the state. MyJustice has been open to discussing the situation and plans to undertake an assessment of intervention options. • Recommendation 39: Incorporate a Do No Harm and conflict sensitivity review into the assessment of programme intervention options in Rakhine State.
7.2 EU value add EU value add is the extent to which the development intervention adds benefits to what would have resulted from EU member states' interventions only in the partner country. The analysis specifically examines to what extent: • The EU brings a capacity above that of other EU member states; • The EU encourages and promotes coordination, complementarity and coherence of other member states to a greater joint effort; and
EU priorities guide the direction and scope of MyJustice to align with other justice programmes implemented by member states.
Finding 30: EU technical support, funding and project management capacities add value to the MyJustice programme approach and implementation The EU Delegation’s project management approach and funding capacity adds value to the MyJustice programme. Both factors have contributed significantly to the ability of the programme to adopt a learning-centred, adaptive programming approach, based on the shared assumption that
25 In August 2017, an outbreak of violence following the attack by Rohingya militants on police posts in Northern Rakhine caused hundreds of Rohingya to flee to Bangladesh. Since then more than 650,000 Rohingya have fled what the UN High Commissioner for Human Rights has described as ‘ethnic cleansing’.
www.integrityglobal.com 39
this approach will enable more effective interventions and is most suitable for the Myanmar context. Delegation staff contributed country specific expertise during the design and early implementation phase of the programme. The EU also supported the design of MyJustice as a community-focused project given the focus of other actors such as the United Nations Development Programme (UNDP) and the United States Agency for International Development (USAID) on Union-level institution reform. The EU has been relatively ‘hands-off’ in terms of technical input during implementation but maintains regular communication, oversight and provides support and guidance to MyJustice as required. The partnership-based relationship has been appreciated by both parties.
Finding 31: EU values technical engagement with MyJustice but more opportunities exist to support alignment and coordination with other EU/member states programmes The EU regularly seeks strategic advice and guidance from MyJustice regarding other programming areas. MyJustice has met frequently with the EU MyPol (police support programme) at the EU’s request to offer strategic advice and share lessons learned. Given the critical role that police play in the justice system and respondent feedback regarding the potential for MyJustice to engage with the police, the British Council and the EU could consider how to increase complementarity of MyJustice and MyPol in future programme iterations (see section 2.1). Such complementarity could strengthen opportunities to realise systemic change within the justice system. The EU’s Safety, Security and CSO expert has also liaised with MyJustice regularly to discuss lessons learned regarding CSO engagement practices in Myanmar. There is limited evidence of other formal programmatic interaction between MyJustice and its partners and other EU programmes. More coordination at a programme and implementation level could strengthen the overall impact of EU-funded interventions within the justice sector. • Recommendation 40: Discuss with the EU current and/or future opportunities for EU- funded programmes such as MyJustice and MyPol to better coordinate to achieve EU desired results.
Finding 32: EU’s positive donor reputation in Myanmar facilitates acceptance of MyJustice amongst key stakeholders According to respondents, there is a generally a positive perception of the EU in Myanmar. The EU is recognised for engaging all stakeholders, including the Myanmar military (Tatmadaw)26, and as being a helpful partner for the Myanmar government. Interviews with government stakeholders validated that the EU is viewed positively as a key donor. One respondent pointed out that the EU/BC partnership was politically beneficial for MyJustice as it enables the programme to overcome the biases of a range of stakeholders who may have specific views about the EU or the British.
7.3 Other justice sector programming Finding 33: MyJustice is well-aligned with other foreign-funded justice sector programmes in Myanmar despite some duplication of activities
26 It is noted that the EU and Members States decided on 16 October 2017 to suspend invitations to the Commander in Chief of the Myanmar Armed Forces and other senior military officers in light of the disproportionate use of force carried out by security forces in Rakhine.
www.integrityglobal.com 40
There are a range of foreign donors and international organisations supporting Myanmar's justice sector development. These include UNDP with International Development Law Organisation (IDLO), USAID, the German Foreign Office with the International Legal Foundation, and the Danish International Development Agency (DANIDA) with the International Commission of Jurists (ICJ) and DIHR. All support or implement broadly similar rule of law and justice-related activities, such as: • Legal education; • Legal aid provision (both civil and criminal); • Legal awareness raising; • Training for justice sector actors including the judiciary, law officers and lawyers; and • Paralegal programmes.
They also tend to engage the same key stakeholder groups. At a national government-level these stakeholders are primarily the OSCU, the UAGO and the Union-level Justice Sector Coordination Body. They also engage non-governmental actors including a range of individual CSOs and CSO networks, such as Access to Justice Initiative, and lawyers’ networks, for example, the Independent Lawyers Association of Myanmar (ILAM). Despite donors supporting similar activities across the sector, such as the recently established the International Legal Foundation (ILF) public defender’s offices in Yangon and Mandalay, several respondents noted that Myanmar is such a large country and has such vast needs, that, as one respondent stated, "every initiative is just a drop in the bucket" (Programme Stakeholder). Within this setting, the MyJustice programme is unique in two main ways: • It is the largest funded A2J programme in Myanmar, allowing it to test or pilot a range of interventions, engage with multiple partners and cover greater geographical areas than other programmes; and • It is the only major A2J programme that explicitly prioritises community-level or ‘bottom-up’ engagement (instead of institutional reform).
Finding 34: MyJustice engages in higher-level coordination with other access to justice stakeholders but opportunities exist to strengthen coordination at the community-level MyJustice prioritises complementarity and attempts to minimise the risks of unnecessary duplication of interventions (both in terms of location and/or substance), as evidenced, for example, by the extensive mapping and consultations undertaken when determining the sites for the new justice centres. The programme seeks to ensure programme coherence with other key stakeholders, such as other foreign-funded access to justice programmes as well as the Myanmar government. MyJustice’s coordination efforts are driven by key principles of engagement and communication, transparency, prioritisation and complementarity. Specific programme coordination activities include: • Participation in forums with other access to justice programmes and government representatives, such as the UNDP-led Rule of Law Monthly Coordination meetings and the Union-level JSCB Advisory Group meetings. • Sharing results of recent research with key external stakeholders, such as other donors, A2J implementers and government officials. • Supporting coordination efforts of other stakeholders, such as its recent support to the government to organise the National Justice Conference.
www.integrityglobal.com 41
• Seeking opportunities to complement implementation activities of other A2J programmes (and vice versa). Examples include: o the collaboration with UNDP on the WVTA training; o coordinating with DANIDA and DIHR in the delivery of legal education at universities; and o the informal collaboration between the new justice centres and the UNDP/IDLO Rule of Law Centres.
As the number of development actors in the justice space continues to increase, coordination to avoid duplication and overlap of target caseloads (for example, with ILF public defender centres) will continue to be important. Respondents also emphasised the need for more community-level coordination amongst implementers of access to justice programmes. Several suggested that MyJustice is well placed to play a greater coordination role, especially due to its large number of partners. MyJustice encourages coordination between its partners at a community-level to ensure coherence, and to increase programme effectiveness, efficiency and relevance. Initiatives such as the Partner's Forum (December 2017), the developing communities of practice, and the Yangon and Mon partner coordination meetings attempt to build relationships between partners that are working in similar geographical locations and/or on similar issues. Partners also see these initiatives as important opportunities to discuss and coordinate activities and approaches. There is evidence of an appetite amongst partners and other stakeholders to participate in MyJustice-led coordination initiatives. Evidence includes: o An eagerness and willingness from lawyers for greater cooperation between Rule of Law Centres and Justice Centres o Lawyers’ interest in further opportunities for multi-stakeholder justice roundtable discussions. o The newly developed community of practice for lawyers. MyJustice should track this as a potential model for enhancing coordination and coherence of approaches. o Some partners emphasised that MyJustice has unique access to a vast array of stakeholders and local organisations. A formal platform that enabled partners to interact more regularly and with a range of stakeholders would have a significant impact on their ability to achieve their objectives. o Some partners specifically requested MyJustice to play a greater role as a bridge between government actors (e.g. JSCB) and communities/CSOs.
• Recommendation 41: Explore jointly with partners, CSOs, and government representatives options for systematising a process of government-community engagement to promote access to justice. • Recommendation 42: Discuss with partners and CSOs opportunities and potential mechanisms for greater community-level coordination, including for example, piloting one or more mechanisms in specific locations.
www.integrityglobal.com 42
8 Conclusions and recommendations MyJustice is committed to a programmatic approach that begins with placing a range of ‘small bets’, learning from doing, and adapting its programme in response to a deepening understanding of the context and of what does or does not work. At the same time, the programme is gathering evidence that tests and enables adaptation at a theory of change-level. Ultimately through this approach, the programme will be able to show which discrete activities were successful, which assumptions are valid in the Myanmar context, and therefore what change is possible and how. This learning has the potential to strengthen future justice sector programming decisions for the EU and British Council, and for the international development community writ large. Many of the elements needed to achieve the programme’s overall objectives are in place; however, MyJustice will struggle to meet these objectives within the remaining timeframe. This is not a weakness of the programme, but a realistic assessment of what change is possible within a four- year period given the complex and dynamic Myanmar context. The programme has the evidence and lessons to know what is working and what needs further adaptation to continue enabling the desired change. However, the programme theory of change ambition needs to be tailored to what is possible within the remaining months of the programme period. In the remaining time, the programme should aim to consolidate its learning and focus on deepening its engagement in existing target locations and with its direct stakeholders. This engagement will ensure grassroots learning informs programme decisions, that the needs of the poor and vulnerable continue to be prioritised, and that the quality of justice services provided is both measured and improved. The following table compiles the review’s recommendations. These are designed to facilitate the process of learning consolidation, to support the programme to achieve its stated objectives in the remainder of the programme period, and to guide future programming decisions.
Table 6: Recommendations and Considerations (by Theme) To enable greater programme relevance to justice needs, context and the theory of change:
Engage with the GESI assessment and work with partners to address and implement 1 these findings at both programme and partner-levels. Review existing partner engagement mechanisms (e.g. partner forums) and internal 2 staffing roles and responsibilities (e.g. field offices) to promote more systematic engagement of partners and sub-partners regarding grassroots justice needs. Undertake analysis of Knowledge, Attitudes and Practice and baseline data to inform 3 programme adaptation and to ensure the programme remains responsive to the justice needs of the poor and vulnerable. Consult with partners and other relevant stakeholders regarding options for engaging the 4 police in a future iteration of the MyJustice programme. Transfer the UAGO English Language Training to Result Area 4 and explore with the 5 teacher how the activity could more explicitly complement policy engagement efforts.
www.integrityglobal.com 43
To support greater efficiency over the remainder of the programme period: Articulate communications support more explicitly, specifically:
Adapt the result area strategic framework tool to include a section for ‘Communications’ 6 that articulates how communications activities support achievement of the result area and programme goals.
Refine contracting and administrative processes, specifically:
Conduct an internal review of the contracting process to ensure (to the extent possible) 7 processes are aligning to appropriate due diligence requirements and clarify roles and responsibilities within this process. Clarify the information that is the most critical for the programme in terms of partner 8 reporting. Inform partners about and focus MyJustice feedback on this critical information. Review partner reporting formats with MyJustice programme and technical teams and 9 partners to develop a more streamlined and efficient format for future grants under the current programme and/or in a second iteration of the programme. Review the change request system in consultation with partners, programme and technical 10 teams, to address perceived and actual barriers to partner-level activity adaptation.
Strengthen team cohesion and prioritise interventions, specifically
Undertake a participatory programme synergy mapping exercise to support MyJustice staff team and partners to understand the wider programme approach and theory, reflect on 11 how resources are currently being used and to what effect, and assist to prioritise activities for Year 4 and inform a potential second iteration of the programme. Identify how best to integrate and utilise the field officers in promoting programme 12 objectives, particularly in relation to increased engagement with partners and sub-partners.
Strengthen the programme’s monitoring systems, specifically:
Set targets across all indicators to ensure progress can be tracked for both accountability 13a and learning processes. 13b Define baseline data sources to ensure comparable endline data can be gathered. Establish whether the final evaluation exercise is intended to contribute to endline data 13c collection, and if so ensure the exercise is sufficient resourced. 13d Define the monitoring responsibilities of MyJustice staff members. 13e Ensure the MEL team undertakes systematic spot checks of partner data to check validity.
www.integrityglobal.com 44
To deliver more effective programming and better track progress towards objectives: Establish monitoring systems to track awareness raising activities and capacity development, specifically:
Prioritise target setting for Indicator 1.2 and communicate this with partners as soon as 14 possible. When setting the target consider the ambition of knowledge change, given the context, quality issues with the initial interventions, and the remaining programme. Utilise tools such as sampled FGDs or community surveys conducted several months 15 following awareness raising activities, to track the impact of these activities on knowledge change. Insert the CSO capacity development baseline and targets into the MEL framework to track 16 progress and assess the result of these activities. Develop an indicator/s to monitor quality of interventions to assess the effectiveness of 17 capacity development of partners, but also to quality control interventions. Develop and apply a clear monitoring framework to assess CSO capacity development 18 and supplement this with interviews and feedback sessions with partners to get their perspectives and inputs on the monitoring methods and tools. Explore funding opportunities for partners that have participated in capacity development 19 activities to continue to apply their new skills in awareness raising activities in target locations.
Review indicators and activity alignment to track improved justice services, specifically:
Review the expected change under Indicator 2.1 to assess if achievement is still possible 20 given the limited complementary services. Revise the logframe if required. Consider revising Indicator 2.5 to strengthen its accuracy. For example, a revised indicator 21 that can also be a proxy for measuring ‘trust,’ could be the number of referrals by former clients. Clarify how the indicator 2.4 is to be assessed and establish a baseline and target to 22 enable monitoring and evaluation. Align the English language training activity to Result Area 4 to more accurately reflect the 23 linkage between the activity results and the intended change.
Consider specific issues related to justice centres, specifically:
24 Review the TJC referral system with TJC staff and lawyers to address barriers to payment. Review the legal aid criteria, existing case data, and relevant baseline data with each justice centre to identify how best the centres can: 25 • respond to the priority justice needs of their communities; and • prioritise legal representation for the most vulnerable
www.integrityglobal.com 45
To generate evidence of the programme’s contribution to impact:
Begin systematically collecting monitoring data on knowledge change in target 26 communities where interventions are already considered ‘good quality’ so as to assess the impact of these interventions on knowledge change. Incorporate questions into Result Area 2 monitoring tools that ask lawyers about other 27 training and support they receive from other programmes to capture potential overlaps and complementarity. Clarify in the MEL framework what skills will be assessed to measure increased capacity of 28 legal aid lawyers in order to measure progress against Indicator 2.4. Undertake an independent justice monitoring case study in Hlaing Thayar that tests the theory of change logic by tracking beneficiaries and focusing on the process - what role did 29 the awareness raising have, how did the justice process work – as well as the result, i.e. was access to justice attained? Consider whether the programme’s engagement with the OSCU on the Legal Aid Law can 30 be leveraged to promote behaviour change of judges and, if so, articulate this in the result area strategic framework and allocate necessary resources. Engage with partners to further test the assumption underpinning CBDR engagement and 31 identify potential barriers to access that can be addressed through partner training and mentorship activities. Establish a system for monitoring how MyJustice research is used and applied by 32 policymakers to their daily work. Such data would usefully inform future research outputs, dissemination strategies and programme follow-up. Engage with the justice centres and other service providers to capture learning on how 33 corruption is affecting their ability to deliver quality services and jointly identify processes to overcome these challenges and/or establish realistic indicators of success.
To promote greater sustainability of activities and change processes:
Continue to engage the EU and British Council in discussions regarding programme 34 sustainability as a matter of urgency, including contingency planning in case future EU funding is unavailable. Continue to empower staff at all levels, and particularly those likely to participate in a 35 second iteration of the programme, to more actively engage in programme and activity- level decision making and representation with government officials. Prioritise support to legacy justice centres to develop funding strategies for continued and 36 diversified donor support beyond the programme period, including for example, proposal writing and donor relations skills development. 37 Engage with all partners to support the development of financial sustainability strategies. Incorporate lessons learned regarding local CSO organisational capacity needs into 38 planning for a second iteration of the programme.
To sustain and generate more coherence:
Incorporate a Do No Harm and conflict sensitivity review into the assessment of 39 programme intervention options in Rakhine State.
www.integrityglobal.com 46
Discuss with the EU current and/or future opportunities for EU-funded programmes such 40 as MyJustice and MyPol to better coordinate to achieve EU desired results. Explore jointly with partners, CSOs, and government representatives options for 41 systematising a process of government-community engagement to promote access to justice. Discuss with partners and CSOs opportunities and potential mechanisms for greater 42 community-level coordination, including for example, piloting one or more mechanisms in specific locations.
www.integrityglobal.com 47
9 Annexes 9.1 MyJustice Logical Framework
Description Objectively Sources and Baseline Target Results to date Assumption (s) / verifiable indicators means of and risks of achievement verification
Improve access to 1. More people have Baseline and 1. None 1. Increase in knowledge, justice and legal the knowledge, endline KAP 2. Year I: 2,108 confidence and aid for the poor confidence and surveys Legal aid - 1,150 opportunities to resolve and vulnerable, opportunities to Legal advice - 567 disputes fairly will be develop legal have their disputes Programme Awareness raising measured using baseline capacity of justice fairly and equitably records - 391 and endline surveys. sector resolved.
Overall objective professionals and strengthen 2. Number of 2. Number of individuals selected rule of individuals directly reached and benefited law institutions to benefitting from from MyJustice better fulfil their MyJustice programme to date: 19,341 mandates. programme (EU (Yr I: 2,108, Yr II: 9,466, result framework indicator) Yr III Q1: 7,767)
• Legal aid – 3,839 (Yr I: 1,150, Yr II: 1,819, Yr III Q1: 870) • Legal advice – 2,192 (Yr I: 567, Yr II: 1,155, Yr III Q1: 470) • Awareness/Training – 13,310 (Yr I: 391, Yr II: 6,492, Yr III Q1: 6,427) • Social Media – Over 75,000 likes on MyJustice Facebook page
www.integrityglobal.com 48
1. To improve 1. 15% increase in Baseline and 1. Year I: 1,717 1. 73% increase in direct Assumptions: access to both number of users endline KAP (Legal aid - 1,150, beneficiaries from That greater formal and from MyJustice surveys Legal advice - MyJustice supported legal awareness, quality informal justice communities 567) aid and legal advice and availability of systems accessing Justice score card 2. Baseline (2017) services. justice services will especially for MyJustice Informal providers lead to greater vulnerable groups supported justice Programme (W/VTA): (Yr I: 1,717, Yr II: 2,974, demand for and use in six services. records Reliability 77% Yr III Q1: 1,340) of these services. Specificobjectives Regions/States in Treat with respect Myanmar 2. 15% increase in 69% There is a will and availability and Unbiased 59% 2. Increase in availability and commitment to 2. To strengthen responsiveness of No unofficial fees responsiveness will be change within the the capacity of justice services. 51% measured using baseline delivery of justice and endline surveys. formal and Affordable 46% services and related informal justice 3. 25% increase in Accessibility 92% practices by all service providers service providers. number of formal 3. Yr I: None, Yr II: 818, to provide Formal providers and informal Yr III Q1: 710) accessible justice justice sector (Judge): services in Reliability 5% providers engaged 484 Officers from General Myanmar Treat with respect Risks: in MyJustice Administration Department 3.8% Overall risk: formal interventions (GAD), 140 lawyers and Unbiased 11.6% and informal justice disaggregated by 194 Ward and Village No unofficial fees service providers type of justice Administrators (W/VTA) 4.3% may not have the service and attended MyJustice Affordable 2.6% will, or the gender. supported trainings in Year Accessibility 5.4% opportunity, to II. change– the rule of
498 GAD Officers, 92 law space is 3. None W/VTAs, and 120 emerging, with community leaders MyJustice navigating attended MyJustice a new political supported trainings in Year sphere. III Q1.
www.integrityglobal.com 49
Result Area 1: 3.1. Number of people Baseline and 3.1. Awareness raising Assumptions: Increased reached through endline KAP 1.1 391 1.1 10,000 That awareness and capacity within MyJustice surveys people reached Yr I: 391, Yr II: 6,492, understanding of through communities to supported Yr III Q1: 5,586 rights, and options understand and awareness raising Communications 1.2 50% can community for accessing justice apply their rights activities. activity records identify rights based will stimulate including freedom, awareness application of this equality and activities and Social Media – Over Expected Results 3.2. % of people with Media monitoring knowledge, resulting protection from approximated 2 75,000 likes on MyJustice increased records Facebook page in greater demand knowledge of abuses. (Baseline million through for more capable, rights among Monitoring reports 2017) other mass accountable & media activities MyJustice from justice 3.2. Increase in knowledge of responsive justice supported centres rights among MyJustice services. communities 1.3 None communities will be Risks 1.2 to be set measured using baseline Raising demand for 3.3. Increased capacity later and endline surveys. and expectations of of MyJustice Baseline survey is in justice services that partners to design process and data will be cannot be met; could and implement 1.3 50% of available by October do more harm than legal awareness those 2017. good within interventions. organisation communities which 3.3. Capacity development (especially for participate in activity for CSOs will be vulnerable groups). MyJustice implemented in Yr III. capacity Increasing demand development does not guarantee activity of legal enhanced political or awareness institutional will for strategy design reform.
Result Area 2: 2.1. Increase in Programme 2.1 24 townships 2.1 50 2.1. Yr I: 24 townships, Assumptions: Justice services number of records townships Yr II: 44 townships Supporting the are more widely townships with 2.2 1,717 Yr III Q1: strengthening of available, of complementary Training records 2.2 12,500 justice services will higher standard, justice services 2.3 None persons Number of townships lead to them and increasingly supported by Baseline and with complementary becoming more meet the needs of MyJustice. endline KAP 2.4 None 2.3 300 justice services responsive to communities and surveys supported by MyJustice people’s needs, and 2.2. Number of people their residents 2.5 None 2.4 – 2.6: to be has increased to 44 in that wider availability accessing justice set later Year II from 24 in Year I will lead to more services provided through MyJustice 2.6 None as a result of CSO people using them. support. grants and new justice centres. Social and cultural 2.3. Number of service barriers that stand in
www.integrityglobal.com 50
providers trained 2.2. Yr I: 1,717, Yr II: 2,974 the way of justice (by MyJustice Yr III Q1: 1,340 service delivery do directly or through Number of direct not disincentivise our partners). beneficiaries from potential users of MyJustice supported justice services. 2.4. Increase in number of justice legal aid and legal service providers advice services increase Risks from 1,717 in Year I to Resistance among supported by 2,974 in Year II. formal actors MyJustice who demonstrate 2.3. Yr I: None, Yr II: 255 (courts, police, improved capacity lawyers) to 140 lawyers and 115 innovative/ (knowledge, paralegals attended reformative practices understanding, MyJustice supported practice) to deliver of MyJustice trainings in Year II. legal aid services partners
for the poor 2.4. Follow-up assessment Obstruction/denial of will be conducted on the 2.5. Increase in level of capacity among access to persons in satisfaction among the service providers custody by police and courts users of MyJustice trained by MyJustice
supported justice 2.5. Follow assessment will services Distrust of be conducted on the MyJustice-supported 2.6. % of clients level of satisfaction legal and paralegal referred to among users of services because MyJustice MyJustice supported they are provided at supported justice justice services. no or low cost centres by duty 2.6. 27% of clients were bearers. referred by police and Legal profession’s judges in Yangon. potential hostility to paralegal activity
Result Area 3: 3.1. Number of CBDR Training records None 3.1,200 3.1. 484 Officers from Inclusive and CBDR actors trained by (W/VTA), General Administration representative mechanisms MyJustice. CSO partner 1,200 (GAD Department (GAD), and community based increasingly utilise reports Officials) and 194 Ward and Village dispute resolution fair approaches 1,000 Administrators (W/VTAs) mechanisms will for dispute 3.2. % of CBDR actors Paralegal case community attended MyJustice improve access to resolution within trained by records and leaders supported CBDR justice for poor and the communities MyJustice who reports trainings in Year II. vulnerable people. they serve. report increased 3.2 50% of the knowledge and Monitoring above 452 GAD Officers, 92
www.integrityglobal.com 51
understanding of checklist W/VTAs, and 120 fair approaches for 3.3 to be community leaders Traditional and State dispute resolution. Justice Scorecard discussed with attended MyJustice appointed village partners and supported trainings in headmen will accept 3.3. % increase of set later Year III Q1. outside intervention CBDR actors in dispute resolution trained by 3.4 to be 3.2. According to pre and in their communities. MyJustice who use discussed with post tests, GAD Officers fair approaches for partners and have a better That social norms do dispute resolution. set later understanding of not preclude civilians distinction between Rule in communities, 3.4. Increase in of Law and Rule by Law particularly women number of and increasingly agree and disadvantaged MyJustice that everyone is entitled groups, from supported CBDR to fair and equal engaging with new users who report a treatment by law approaches to fair approach regardless of social resolving disputes. being used in the background. process of their Risks: dispute resolution. The post evaluation of Existing community W/VTA trainings based dispute indicated a higher level resolution of tendency among mechanisms and W/VTAs to act more as actors will not accept facilitator rather than MyJustice supported decision maker in the approaches. dispute resolution process. Discriminatory practices are 3.3. Follow-up assessment inadvertently will be conducted on adopted into new approaches used by dispute resolution CBDR actors trained. approaches.
3.4. Follow-up assessment Dispute resolution is will be conducted with used inappropriately CBDR users if fair to resolve or approaches have been mediate cases that used in dispute are the clear remit of resolution process. the criminal justice system.
Local elites capture new mechanisms of
www.integrityglobal.com 52
dispute resolution for rent-seeking opportunities.
Obtaining quality CBDR case tracking and resolution data
Result Area 4: 3.1 Number of Minutes and None 4.1 5 research 3.1 MyJustice’s Access to Assumption: Increased MyJustice attendance sheets / analysis Justice research report Robust, better evidence and research and from policy products was published in informed policy knowledge analysis products dialogues and November 2016. frameworks will sharing to inform that inform the briefings 4.2 70% of the 3.2 Capacity development facilitate better justice policy programme’s organisations promotion and activity for CSOs will be development and policy engagement Reports and which protection of implemented in Yr III. implementation and broad records (including participate in rights/access to audience. visual records) of MyJustice 3.3 MyJustice has been justice for vulnerable engagement capacity working closely with people. 3.2 % of MyJustice opportunities and development newly established partners that knowledge activities on Justice Sector Risks: demonstrate sharing events policy Coordination Body at Lack of willingness increased capacity advocacy both union and to reform within the for policy Minutes and State/Region levels. Rule of Law engagement. attendance of 4.3 15 Institutions, and workshops or strategic MyJustice regularly receptivity to 3.3 Number of presentations engagement attended JSCB consider and review strategic activities meetings at the Union evidence in policy engagements Records for the undertaken to level and met with decision making. MyJustice receipt of research share State/Region JSCBs undertakes to reports and MyJustice from Mon, Kayin and inform policy communications evidence and Bago in August 2017. development and products inform policy MyJustice KAP implementation on findings were access to justice Publication on 4.4 to presented to Supreme issues. MyJustice website discussed with Court, UAGO, MOHA, 3.4 Number of partners and Yangon Region JSCB MyJustice Key Informant set later and Law Officers in Interviews (KIIs) knowledge sharing Year III Q1. with stakeholders events, briefings to gauge 3.4 MyJustice’s Access to and policy receptivity of Justice research dialogues where findings were information, skills evidence generated and presented in Yangon, and evidence from knowledge shared Mandalay,
www.integrityglobal.com 53
MyJustice by MyJustice Mawlamyine and programme are Taunggyi, in presented, shared Analysis of law November 2016, and discussed. and regulation March 2017 and April around Legal 2017 to which local Framework. justice actors have attended. CSO Advocacy Index MyJustice also shared Access to Justice research findings with the UGAO, Legal Affairs and Special Cases Assessment Commission, MPs from Yangon and Mandalay and Justice Sector Coordination Body.
MyJustice hosted policy dialogue forum with Myanmar policy/research institutes to discuss policy reform on justice at the British Council Library in Yangon in June 2017.
MyJustice baseline findings were presented at Partners' Forum in December 2017.
www.integrityglobal.com 54
9.2 Review Framework
Criteria & Reasons for change OECD-DAC TOR Guidance Review Questions communicated to MyJustice Definition
Changes to Q1: Slight change to Relevance This relates primarily to Q1: How relevant is was the original programme concept and objectives to the wording to focus on whether the justice needs of targeted beneficiaries? The • programme preparations, current programme is relevant to the programme is design and the extent to • Q1.1 How did the programme identify justice needs? concept and objectives. This will build suited to the which the stated objectives on previous reviews that have been priorities and coherently address the • Q1.2 How did those needs inform programme approach, objectives, completed and focus on the current policies of the identified problems or real and design? programme. target group, needs; Q2: To what extent has the programme responded to changes in the Myanmar recipient and -- • the responsiveness of the context and priorities of the Myanmar government? and priorities of key donor. stakeholders (e.g. EU and Myanmar government)? programme to the physical Changes to Q2: This question will and policy environment i.e. • Q2.1 What has changed in the Myanmar context (as it relates to focus on context changes related to main political, social and access to justice) since the beginning of MyJustice? the Myanmar context and relevant economic factors as well as government policy. We have removed geographical coverage of • Q2.3 Are there examples of the programme adapting to these reference to the EU as this is covered the programme and the changes? in Q17. pertinent policies/positions • Q2.4 If the programme has not responded to changes, why? -- of various stakeholders including the government, Q3: To what extent are MyJustice activities consistent with its stated objectives Changes to Q3: Changed ‘impact’ to the EU, other donors and and impact theory of change? theory of change as the focus will be interest groups; understanding the process in which • Q3.3 Are there instances of MyJustice undertaking activities that are the programme aligns activities with • the consistency of the not consistent with the programme’s stated objectives and theory of the overarching theory of change. The activities in relation to the change? ToR requirements emphasise an overall goal and attainment assessment of the extent to which of objectives; consistency of • Q3.4 If yes, why has the programme decided to undertake them? activities are consistent with the activities with intended Q4: What actions can MyJustice take in the residual programme period to stated objectives of the programme. impact and effects. strengthen the programme’s relevance? We will be using a process approach to address this. --
No refinement to Q4
www.integrityglobal.com 55
Criteria & Reasons for change OECD-DAC TOR Guidance Review Questions communicated to MyJustice Definition
Slight wording adjustments to Q5 Efficiency This concerns how well the Q5: In what way has MyJustice’s internal structures and processes supported various activities of the or acted as a barrier to the programmes implementation and progress towards Measurement -- programme transformed the its stated objectives? of outputs in available resources into relation to the • Q5.41 How has the learning adaptive approach to programming (e.g. No refinement to Q6 intended results in terms of inputs. The quantity, quality, value for Problem Driven Iterative Adaptive, or PDIA learning and adaptive programme money and timeliness. This programming) facilitated or inhibited implementation? uses the least includes a review of the • Q5.32 How have internal MyJustice programme structures processes costly programme’s governance and resources (e.g. management, programme, MEL, and Communications teams, management structure, with PAC) facilitated or inhibited implementation possible to attention paid to the PAGoDA achieve the modality, and the chosen • Q5.13 How has indirect management offered by the PAGoDA funding desired intervention methods and how modality facilitated or inhibited MyJustice implementation? results. monitoring and evaluation has • Q5.24 How have British Council administrative processes facilitated or been designed and further inhibited implementation? carried out. Attention should be paid to the problem analysis • Q5.5 How have MyJustice processes (e.g. MEL, contracting, grants undertaken as part of management) facilitated or inhibited implementation? programme review exercises and how this informs • Q5.6 How has the programme’s response to its partners’ needs (as programme design and identified in activity monitoring) facilitated or inhibited implementation? iterations. Q5.6 How has the programme’s response to its partners’ needs (as identified in activity monitoring) facilitated or
Q6: What actions can MyJustice take in the residual programme period to strengthen the programme’s efficiency?
No changes to Q7. Note that the Effectiveness This component concerns the Q7: Are the results of the MyJustice activities to date likely to lead to its analysis will focus on what evidence extent to which the programme achievement of its objectives in the programme’s remaining time? The extent to exists that activities are leading to results have contributed results and whether this evidence is which the towards the achievement of the • Q7.1 What are MyJustice’s key results (both intended or unintended) programme to date? convincing. purpose (including a -- has attained consideration of the its objectives. • Q7.2 Which desired results (as stated in the logframe) are behind assumptions) or whether it can schedule? Why? Changes to Q8: Overarching be expected to happen in future question changed to address the ToR Q8: What external factors have supported or acted as barriers to the on the basis of the current guidance more closely which is www.integrityglobal.com 56
Criteria & Reasons for change OECD-DAC TOR Guidance Review Questions communicated to MyJustice Definition
results of the programme. implementation of MyJustice interventions? Have the programme assumptions focused on a review of assumptions. Attention should also be paid to held true? -- results or effects – both positive and negative – not included in No refinement to Q9 the original/revised intervention Q9: What actions can MyJustice take in the residual programme period to logic. increase the chances of achieving the expected results? Attention also needs to be paid to whether the assumptions, required to translate programme results into purpose, have been realised and if not, whether the assumption are likely to be realised in future.
Changes to Q10: Propose to remove Impact Assess the positive and Review Team Note: As this is a Mid-Term Review, we will focus on assessing this question from the MTR questions. negative changes produced by the evidence of early signs of change and progress towards its main impact Direct/indirect After a review of available data, it has the programme so far, directly objectives. become apparent that there is no and or indirectly, either intended or intended/unint Q10: Is there evidence of changes in the knowledge and practice of justice consistent existing data available to unintended. This concerns answer this question. We will not have ended results what is happening as a result sector actors service providers and users (intended or unintended) that have of the resulted from MyJustice programme activity to date? the scope in our MTR to conduct of the implementation of the enough interviews across the relevant programme, programme, how many people including the • Q10.1 Is there evidence of changes in the knowledge and practice of result areas to answer this thoroughly. are being reached and the justice service providers? -- main impact difference the activities are resulting from making in people’s lives and if • Q10.2 Is there evidence of changes in the knowledge and practice of No changes made to Q11. the the programme’s approach legal aid users? programme on -- evolving in response to Q11: Does evidence exist that MyJustice programme activities are beginning the local evolving needs. social, to enable greater access to more inclusive, accountable, and responsive No changes made to Q12. economic, justice services for people in programme locations, and especially the most environmental vulnerable. and other Q12: What actions can MyJustice undertake to increase the chances that it will development contribute to its overall objective? indicators. Include the positive and negative impact of
www.integrityglobal.com 57
Criteria & Reasons for change OECD-DAC TOR Guidance Review Questions communicated to MyJustice Definition external factors on the programme.
Changes to Q13: Second sub- Sustainability The sustainability criteria will Review Team Note: As this is a Mid-Term Review, we will focus on assessing question has been adjusted. Based look into whether the benefits the evidence of early signs of sustainability, such as evidence of contingency on the evaluability assessment, we The benefits of of the programme are likely to planning, an exit strategy, and identifying other funding opportunities. the don’t anticipate data exists that has continue after the been tracking the knowledge and programme implementation period and Q13: What evidence is there that justice service providers supported by are likely to MyJustice are beginning to plan for ways to provide inclusive, accountable, skills of service providers. As such, current funding comes to an the original sub-question will not be continue after end. and responsive services in the absence of the programme’s support? donor funding possible to answer. Instead, we have is withdrawn. It should include a discussion • Q13.1 Is the programme working with service providers to develop a shifted the focus on whether service of the factors influencing the feasible funding strategy following the end of MyJustice funding? providers ‘perceive’ themselves to
achievement or non- • Q13.2 Are service providers beginning to exhibit knowledge and skills have the skills and knowledge needed. achievement of potential from participation in MyJustice that they will be able to use after their sustainability of the participation in the programme? Do service providers perceive that -- programme. they have the knowledge and skills needed to continue their activities after the MyJustice programme ends? No changes to Q14. • Q13.3 Is MyJustice engaging with partners who have internal sustainability strategies and/or are expected to continue their operations in the foreseeable future? If not, is it helping them to formulate strategies?] Q14: What actions can MyJustice take in the residual programme period to strengthen the sustainability of the activities it supports?
EU-Value Add This group of criteria is related Q15: What added benefit is the EU providing through MyJustice that would not No changes to Q15 to the principle of subsidiarity have been possible with bilateral funding from the Member States? The and relates to whether the -- programme Q16: What actions can the EU and MyJustice take in the residual programme activity has added benefits to No changes to Q16 activity has what would have resulted from period to strengthen the EU’s value add? added benefits an intervention implemented to what would just by any of the Member have resulted states. from Member States’ interventions
www.integrityglobal.com 58
Criteria & Reasons for change OECD-DAC TOR Guidance Review Questions communicated to MyJustice Definition only.
Coherence The coherence criteria will Q17: To what extent is the MyJustice programme objectives and activities Changes to Q17: Main question to examine the extent to which aligned with the EU and Myanmar government policies and priorities? just focus on EU policies to address The extent to the activities are in sync with ToR requirement and keep analysis which the European Union Q18: To what extent does the MyJustice programme align, rather than of Myanmar government policies to activities allow Delegation’s (EUD) duplicate, other justice sector programming in Myanmar? Q2. the EU to development policies and do Q19: To what extent do the activities of its partners align with the programme’s achieve its not contradict other EU -- development objectives and approach? policies. The complementarity No Changes to Q18 policy with partner country initiatives • Q19.1 Do partner’s activities align with MyJustice objectives? objectives -- or those of other donors to • Q19.2 Do partner’s activities align with MyJustice approach? without demonstrate how the different internal Changes to Q19 actions are mutually • Q19.3 What if any impact has any lack of coherence between the contradiction reinforcing or not should be partner and programme’s objectives and approach had on • Removed since analysis of or assessed. The connection of implementation? consistency of activities to contradiction the action is likely to contribute objectives will be with other EU to or contradict other EU Q20: What actions can the EU and MyJustice take in the residual programme incorporated into Q3. policies. policies should also be period to strengthen the EU’s value add coherence of the programme? -- Extent to assessed. which they Changes to Q20: ‘EU value add’ complement replaced by ‘coherence’ as ‘EU partner value add’ is addressed in Q16. country’s policies and -- other donor initiatives.
www.integrityglobal.com 59
9.3 Mid-Term Review Terms of Reference
MyJustice: Enhancing Access to Justice for the People of Myanmar
MID-TERM REVIEW
TERMS OF REFERENCE (DRAFT)
A: INTRODUCTION AND BACKGROUND
The European Union has contracted the British Council to deliver the rule of law and access to justice component of its Strengthening Governance in Myanmar/Burma Programme (DCI- ASIE/2014/037-333) component, through a delegation agreement for the implementation of the MyJustice programme.
A key component of the EU Programme is to support the rule of law and access to justice, to help ensure that citizens are better protected by the law, and to increase their access to formal and informal dispute resolution and justice systems. It will also support civil society in their provision of legal education and awareness-raising for citizens and officials and improve the work of institutions, which support the rule of law.
MyJustice aims to equip the people of Myanmar with the knowledge, confidence and opportunities to resolve conflicts fairly, equitably and justly. Working together with local communities, the programme seeks to have a lasting impact on the way disputes are resolved and justice is delivered, especially for the poor, vulnerable and marginalised. MyJustice seeks to achieve the following:
Overall Objective: Improve access to justice and legal aid for the poor and vulnerable, develop legal capacity of justice sector professionals and strengthen selected rule of law institutions to better fulfil their mandates.
Specific Objectives: 1. To improve access to both formal and informal justice systems especially for vulnerable groups in six regions/states in Myanmar; 2. To strengthen the capacity of formal and non-formal justice service providers in Myanmar. MyJustice places an emphasis on a people-centred approach, which is designed to encourage learning, trust and collaboration among and between all stakeholders. The programme works in close consultation with communities to promote enhanced and equitable justice provision for all groups, especially those currently most excluded. The programme delivers work across four interconnected results areas:
Result Area 1: Increased capacity within communities to use and apply knowledge and understanding about their rights
www.integrityglobal.com 60
Result Area 2: Justice services are more widely available, of higher standard and increasingly meet the needs of communities and their residents. Result Area 3: Community based dispute resolution mechanisms increasingly utilise inclusive and accountable approaches and methods within the communities that they serve Result Area 4: Increased evidence and knowledge sharing to inform justice policy development and implementation.
MyJustice contracted a Learning and Evaluation partner, INTEGRITY27, in November 2016 for applying a critical lens to the programme, to compile learning and undertake external evaluations of the programme.
In the Programme’s inception phase, INTEGRITY conducted a review of the MyJustice Monitoring, Learning and Evaluation (MEL) strategy and made recommendations, provided critical feedback on its learning and evaluation framework and provided insights in the design of various research pieces commissioned through the programme. INTEGRITY engages in an on-going critical enquiry of the programme design, the problems the programme is trying to address and whether it is doing so effectively and what it is learning in the process. INTEGRITY is not engaged in monitoring the programme or gathering learning at the activity level. MyJustice requests INTEGRITY to undertake the mid-term review of the Programme as a formative evaluation in accordance with the terms set out below. INTEGRITY should develop the methodology, proposed evaluation questions and specific areas of enquiry.
B: DESCRIPTION OF THE ASSIGNMENT
B.1 Global objective
The overall objective of this assignment is to conduct a Mid Term Review (MTR) in order to serve as a formative evaluation of the progress of MyJustice against its anticipated results as specified in the programme’s logical framework (logframe).28
The primary purpose of the MTR is to evaluate the achievements of MyJustice to date and provide recommendations and any adaptations required for the remaining period of implementation and beyond. The review should pay particular attention to the EUD Results Oriented Mission (July, 2017) report and record the response of the programme so far to specific recommendations. In line with the British Council’s approach to evaluation, this MTR of the programme’s overall performance
27 Integrity is a UK based consultancy firm for research and evaluation who has worked in a number of conflict- affected areas and countries in transition, including Syria, Democratic Republic of Congo and South Sudan for evaluation of access to justice programmes. 28 Appended to Annual Workplan 2017-2018 and revised in July 2017.
www.integrityglobal.com 61
against its anticipated results should be conducted using the following OECD/DAC and EU evaluation criteria and methodology29:
1. Relevance: This relates primarily to programme preparations, design and the extent to which the stated objectives coherently address the identified problems or real needs; the responsiveness of the programme to the physical and policy environment i.e main political, social and economic factors as well as geographical coverage of the programme and the pertinent policies/positions of various stakeholders including the government, the EU, other donors and interest groups; the consistency of the activities in relation to the overall goal and attainment of objectives; consistency of activities with intended impact and effects. 2. Efficiency: This concerns how well the various activities of the programme transformed the available resources into intended results in terms of quantity, quality, value for money and timeliness. This includes a review of the programme’s governance and management structure, with attention paid to the PaGODA modality, and the chosen intervention methods and how monitoring and evaluation has been designed and further carried out. Attention should be paid to the problem analysis undertaken as part of programme review exercises and how this informs programme design and iterations. 3. Effectiveness: This component concerns the extent to which the programme results have contributed towards the achievement of the purpose (including a consideration of the assumptions) or whether it can be expected to happen in future on the basis of the current results of the programme. Attention should also be paid to results or effects – both positive and negative – not included in the original/revised intervention logic. Attention also needs to be paid to whether the assumptions, required to translate programme results into purpose, have been realised and if not, whether the assumption are likely to be realised in future. 4. Impact: Assess the positive and negative changes produced by the programme so far, directly or indirectly, either intended or unintended. This concerns what is happening as a result of the implementation of the programme, how many people are being reached and the difference the activities are making in people’s lives and if the programme’s approach evolving in response to evolving needs. 5. Sustainability: The sustainability criteria will look into whether the benefits of the programme are likely to continue after the implementation period and current funding comes to an end. It should include a discussion of the factors influencing the achievement or non-achievement of potential sustainability of the programme. 6. Coherence: The coherence criteria will examine the extent to which the activities are in sync with the EUD’s development policies and do not contradict other EU policies. The complementarity with partner country initiatives or those of other donors to demonstrate how the different actions are mutually reinforcing or not should be assessed. The connection of the action are likely to contribute to or contradict other EU policies should also be assessed. 7. EU added value: This group of criteria is related to the principle of subsidiarity and relates to whether the activity has added benefits to what would have resulted from an intervention implemented just by any of the Member states.
INTEGRITY is required to carry out the MTR as a formative evaluation of the MyJustice programme in accordance with the seven criteria set out above. The evaluation team should pay particular attention to relevant cross cutting issues such as human rights, gender, good
29 http://www.oecd.org/development/evaluation/dcdndep/41612905.pdf (update new link)
www.integrityglobal.com 62
governance, environment, land rights and others to assess the extent to which these factors have been taken into account in the identification/formulation documents and the extent to which they have been reflected in the implementation of the project and its monitoring.
The evaluation team is requested to:
1. Consult with the MyJustice team in September 2017 to develop the evaluation questions, methodology and approach for the mid-term review (MTR).
2. Consult with the EUD in September 2017 on the proposed MTR criteria and questions.
3. Share a powerpoint presentation of the evaluation plan and design during an end of trip debrief in September 2017 with the MyJustice team and EU Delegation for feedback and finalisation.
4. Prepare a written plan for the MTR, with detailed evaluation questions, methodology and approach including a timeline for completion of the MTR, which shall be agreed by the MyJustice programme, British Council and the EUD, prior to commencement of the evaluation. This will be treated as an inception report for the MTR.
5. Conduct the MTR commencing in January 2018, including the following steps:
a. Review all relevant programme documents, background literature, the annual learning review reports, programme management records, the ROM mission report and any other documents deemed necessary for the MTR. b. Interview MyJustice team members, relevant implementing partners, counterparts, stakeholders and programme beneficiaries in MyJustice programme areas in selective states and regions, identified in consultation with MyJustice programme staff c. Interview national justice actors and stakeholders and key development partners in the rule of law and access to justice fields in Myanmar. d. Prepare a first draft report of preliminary MTR findings and present these to the MyJustice programme team, British Council and EUD during the MTR mission during a half-day meeting facilitated by the MTR team to obtain initial comments and feedback. e. Submit a final MTR report incorporating feedback from the MyJustice programme team, British Council and the EUD.
1. A draft Inception Report, submitted to MyJustice no later than 15th September 2017 and a final inception report, incorporating feedback no later than 28 September 2017
2. A Draft MTR Report, submitted to MyJustice no later than 16 February 2018
3. A Final Report submitted to MyJustice no later than 15 March 2018, containing the following structure:
a. Preamble b. Executive Summary c. Relevance of the programme d. Programme Preparation and Design e. Efficiency f. Effectiveness
www.integrityglobal.com 63
g. Overall Outcome and Impact h. Overall Quality/Sustainability/Replicability i. Overall coherence and EU added value j. Conclusions and Recommendations k. Lessons learned
4. Annexes to the MTR Report including:
a. Logframe with suggested revisions b. A map of the programme areas c. MTR Terms of Reference d. Curriculum Vitaes of of evaluators e. Consultant comments on the MTR Terms of Reference, if any f. Methodology used g. List of persons/organisations consulted h. List of programme documents and literature reviewed i. Any other technical annexes deemed necessary
INTEGRITY is expected to deploy the following type of expertise and time inputs to deliver the MTR:
Duration Designation Type of Expertise (working days)
Inception phase (September 7 days30 2017) 1 Main mission phase (January- 15 days March 2018)
Main mission phase (Jan-March 2 20 days 2018)
Main mission phase (Jan-March 3 10 days 2018)
It is expected that the field research and presentation of draft findings for the assignment will be conducted in Myanmar, primarily in Yangon and Nay Pyi Taw but with field visits to partner implementation areas, to be decided in conjunction with the MyJustice team.
30 Inception phase period is already covered by INTEGRITY/MYJUSTICE call down contract 2. The remainder of the assignment will be covered by a further call down contract.
www.integrityglobal.com 64
1. Content The content of the reports required for this assignment are described in the Deliverables section above.
2. Language All reports and correspondence must be in English.
3. Submission/comments timing All reports must be submitted by the MTR Team Leader electronically to the MyJustice Team Leader in the format provided at the start of the mission. Comments by MyJustice and key stakeholders on the various reports will be provided to INTEGRITY as soon as possible.
a. Prepare a MTR Plan no later than three days after completion of inception visit. b. Prepare a final draft report, to be submitted electronically, no later than one week after the end of the field phase. c. Comments to this draft will be provided no later than 20 working days after receipt of the final draft report, which will be incorporated into the final report by the MTR team.
www.integrityglobal.com 65
www.integrityglobal.com 71
www.integrityglobal.com 74
www.integrityglobal.com 76 www.integrityglobal.com 78
9.5 List of organisations consulted Partners and sub-partners: 35
No. Organisation Interview Date 1 ActionAid Myanmar (3) 26th January 2018 2 Arts for All (2) 25th January 2018 3 Colors Rainbow (2) 13th February 2018 4 Danish Institute for Human Rights 14th February 2018 5 International Bridges to Justice (4) 6th February 2018 6 Karen Development Network (5) 2nd February 2018 7 Kings and Queens (4) 25th January 2018 23rd January 2018 8 Local Resource Centre (2) 2nd February 2018 9 Mercy Corps (2) 31st January 2018 10 Mon Women’s Organisation (2) 2nd February 2018 11 Namati 6th February 2018 30th January 2018 12 Norwegian Refugee Council (2) 5th February 2018 13 Terre des Hommes (2) 26th January 2018 14 Yangon Justice Sector Coordination Body (2) 30th January 2018 15 Yaung Chi Oo Workers Association 5th February 2018
Implementers: 7
No. Organisation Interview Date 31st January 2018 16 British Council (3) 6th February 2018 15th February 2018 29th January 2018 17 United Nations Development Programme (3) 5th February 2018 18 USAID/Promoting Rule of Law Project 2nd February 2018
Service Providers: 36
No. Organisation Interview Date 19 Colors Rainbow 13th February 2018 20 Independent 2nd February 2018 31st January 2018 21 Institute for Development Administration (2) 14th February 2018 22 Kings and Queens (3) 13th February 2018
www.integrityglobal.com 79
23 Legal Sunshine Legal Services 10th February 2018 24 Mercy Corps (3) 5th February 2018 25 Ratana Metta Organisation (2) 13th February 2018 26 Taunggyi Justice Centre (4) 7th February 2018 27 Taunggyi Rule of Law Centre (6) 10th February 2018 5th February 2018 13th February 2018 28 Yangon Justice Centre (13) 14th February 2018 15th February 2018
Donors: 2
No. Organisation Interview Date 29 EU Myanmar (2) 24th January 2018
Programme: 22
No. Team Interview Date 24th January 2018 25th January 2018 30 MyJustice (18) 29th January 2018 2nd February 2018 30th January 2018 31 Programme Advisory Committee (4) 1st February 2018
National Government: 4
No. Organisation Interview Date 32 Justice Sector Coordination Body 29th January 2018 33 Office for the Supreme Court of the Union (2) 29th January 2018 34 UK Department for International Development 30th January 2018
Other: 2
No. Organisation Interview Date 22nd January 2018 35 Independent (2) 10th February 2018
www.integrityglobal.com 80
9.6 List of key programme documents reviewed
AAM Quarterly Narrative Report 1, ActionAid Myanmar (2017) AAM Quarterly Narrative Report 2, ActionAid Myanmar (2017) AAM Quarterly Narrative Report 3, ActionAid Myanmar (2017) Access to Justice and Informal Justice Systems Consolidated Summary Report, UNDP (2017) AFXB Quarterly Narrative Report 2, The Association François-Xavier Bagnoud (2017) Annual Learning Review Report, Integrity (2017) Annual Progress Report Year 1 2015-2016, MyJustice (2016) Annual Progress Report Year 2 2016-2017, MyJustice (2017) Appendix 2: Theory of Change, MyJustice (2015) Baseline Report : Knowledge, Attitudes & Practices: Accessing Justice in Myanmar, KANTAR and MyJustice (2018) Biannual Report – Final, MyJustice (2017) British Council Policy and Systems Issues, British Council Capacity Development Strategy for CSO Partners, MyJustice (2017) Considerations for Siting New JCs, MyJustice December 2017 Revised Logframe (with targets), MyJustice (2017) Definitions of MJ Terminology, MyJustice Empowered Hlaing Thar Yar Resident Brings Paralegal Services to Community, European Union (2017) EQMM Quarterly Narrative Report 1, Equality Myanmar (2017) EQMM Quarterly Narrative Report 2, Equality Myanmar (2017) Full Staff Organogram, MyJustice Guidance for Submission of Concept notes, MyJustice (2016) Inception Meeting on Designing Strategies for Legal Awareness, MyJustice (2017) Justice Centre Expansion Grant Technical Proposal, MyJustice (2017) KHRG Quarterly Narrative Report 2, Karen Human Rights Group (2017) Knowledge, Attitudes and Practices (KAP) survey: Justice needs and experiences in Myanmar, MyJustice (2017) KNQ Quarterly Narrative Report 2, Kings N Queens (2017) Leaders' Stories - Interest-Based Mediation training of Ward and Village Tract Administrators in Myanmar, Mercy Corps (2017) Legal Aid in Myanmar: The Yangon and Mawlamyaing Justice Centres, Mercurio, D. (2016) LRC Quarterly Narrative Report 2, Local Resource Centre (2017) Making Big Cases Small and Small Cases Disappear, MyJustice (2016)
www.integrityglobal.com 81
MEL Framework draft, MyJustice (2017) Mercy Corps Quarterly Narrative Report 2, Mercy Corps (2017) MJC Narrative Report July 2016, Mawlimyine Justice Centre (2016) MJC Narrative Report November 2016, Mawlimyine Justice Centre (2016) MJC Narrative Report September 2017, Mawlimyine Justice Centre (2017) Monitoring, Evaluation and Learning Workshop for MyJustice Grantees – Internal Report, MyJustice (2017) Multiannual Indicative Programme (2014-2020) Myanmar/Burma, European Union (2013) MyJustice – Key Lessons, MyJustice MyJustice Grant Mechanism: Reflections on the first round and considerations for a second round, MyJustice (2017) MyJustice Results-Orientated Monitoring Comments, European Commission (2017) MyJustice Results-Orientated Monitoring Questions, European Commission (2017) MyJustice Results-Orientated Monitoring Report, European Commission (2017) Narrative Report Annexes 1-8, MyJustice (2017) NRC Quarterly Narrative Report 2, Norwegian Refugee Council (2017) Options for Implementing New Justice Centres, MyJustice Paralegal Concept and Pilot Intervention Design Report, DIHR (2016) Partner Profiles, MyJustice (2018) Partner’s Forum: Let’s Talk Justice, MyJustice (2018) Programme Management Board - Y2 Q1 Progress Report, MyJustice (2016) Programme Management Board - Y2 Q2 Progress Report, MyJustice (2017) Programme Management Board - Y2 Q3 Progress Report, MyJustice (2017) Programme Management Board – Y3 Q1 Progress Report, MyJustice (2017) Promoting the Rule of Law in Myanmar – Statement of Work Section C, USAID Quarterly Reflection Session Summary, MyJustice (2018) RASF Master File, MyJustice (2017) TdH Quarterly Narrative Report 1, Terre des Hommes (2017) TdH Quarterly Narrative Report 2, Terre des Hommes (2017) TdH Quarterly Narrative Report 3, Terre des Hommes (2017) Theory of Change Diagram, MyJustice Y3 Q1 Data, MyJustice (2018) YCOWA Quarterly Narrative Report 3, Yaung Chi Oo Workers Association (2017) Year 3 Annual Workplan, MyJustice (2017)
www.integrityglobal.com 82