PROJECT PERFORMANCE REVIEW TECHNICAL ASSISTANCE FOR
SOCIOECONOMIC DEVELOPMENT THROUGH DEMINING AND
INCREASING BORDER SURVEILLANCE CAPACITY AT THE EASTERN BORDERS
OF TURKEY
December 12th 2017 CONTENTS EXECUTIVE SUMMARY ...... 2 INTRODUCTION ...... 3 1. MINE ACTION IN TURKEY ...... 5 Landmines and Border Management ...... 5 International Engagement in Mine Action ...... 7 Land Mine Casualties in Turkey: ...... 9 2. STRATEGIC POSITIONING, CONCEPT AND DESIGN...... 10 Project Planning ...... 11 Pre- Tender Planning...... 12 Monitoring, Evaluation and Risk Management ...... 13 The Clearance Contract ...... 14 The Clearance Tender ...... 14 3. IMPLEMENTATION ...... 16 The Establishment of TURMAC ...... 16 Mine Clearance ...... 17 Phase One ...... 18 Phase Two ...... 18 Contracting Changes ...... 20 Reviewing the Contracting Model ...... 20 The Operational Demining Process ...... 21 4. CONCLUSIONS AND RECOMMENDATIONS ...... 23 Project Assessment ...... 23 Conclusions and Recommended Actions ...... 24 Strategic Positioning, Concept and Design ...... 24 Implementation ...... 25 Partnership and Coordination...... 26 Monitoring, Evaluation and Risk Management ...... 27 Rights Based Approach and Gender Mainstreaming ...... 27 ANNEX ONE ...... 29 ANNEX TWO ...... 37 ANNEX THREE ...... 42 ANNEX FOUR ...... 43
Page | 1
EXECUTIVE SUMMARY The Government of Turkey has been enhancing its border surveillance capability in response to the challenges presented by mass migration, terrorism, human trafficking and smuggling. This is a national priority and an essential step in Turkey’s accession to the European Union (EU). The United Nations Development Programme (UNDP) has been supporting the Government by developing and implementing a project entitled “Technical Assistance for Socioeconomic Development through Demining and Increasing the Border Surveillance Capacity at the Eastern Borders of Turkey” in partnership with the Government and the EU. The project was designed to contribute to the prevention of illegal migration and cross-border crime on Turkey’s eastern borders by clearing the border regions of mines and providing an effective and modern border surveillance system. The eastern border areas in Turkey are a military zone and civilians are not permitted to access the area. The area is expected to remain a military zone for the foreseeable future.
The project experienced challenges during the preparatory stages when generous funding was committed to support the clearance of 225 minefields in 24 months. This decision was unfortunately based on insufficient surveying of the minefields and in response a Request for Proposals (RFP) was put out to tender and the successful bid came from a clearance contractor who nominated an unrealistically low cost. A quality management firm was also engaged to ensure that the contract was implemented in an efficient and effective manner and UNDP engaged a small team led by an international Chief Technical Advisor to manage the project.
This was the first humanitarian mine action project that was implemented in Turkey and it quickly experienced some major challenges. The essential governance elements of a national mine action programme such as a National Mine Action Authority and Mine Action Centre were required. The Government, with support from UNDP, established the Turkish Mine Action Centre (TURMAC) although an authority is yet to be established. The establishment of the TURMAC led to the development of core national standards, the clearance contractors being accredited, and legislation being approved to facilitate the transport, storage and use of explosives and other essential activities. After a rocky start the project successfully cleared 25,000 mines and 11 minefields in its first full year of operations in 2017 which has established a sound foundation for future activities.
UNDP and its partners can feel justifiably optimistic ahead of the 2018 demining season and this review is recommending a number of actions to make the programme more efficient and effective. Many of these actions will need to occur in 2019 when the current clearance contract is concluded although planning for 2019 should begin in early 2018. The actions include focusing on the development of an integrated work plan for 2018 with the mine action assets providing a targeted service to the broader border management project. This shift will involve an enhanced coordination and planning role for the TURMAC and link UNDP’s capacity development activities more closely with demining operations.
TURMAC has indicated that it will be moving its Operations/Quality Management and Information Management capability to the minefields in 2018. This will fast track the capacity development of TURMAC staff as they will work alongside the quality management contractor and UNDP Technical Staff. As a part of this shift it is critical that these TURMAC staff are empowered to make operational decisions in the field. These shifts will require UNDP and TURMAC to enhance the results based planning frameworks in the mine action programme to more accurately assess outcomes.
This is an exciting moment for mine action in Turkey as a united group of national and international partners build the capability to eventually rid Turkey of landmines.
Page | 2
INTRODUCTION Background The Government of Turkey in the course of progress towards accession to the European Union (EU) and in response to the obligations of the EU and its Member States is actively following a National Programme for the Adoption of the Acquis. The objective of the process to implement the relevant amendments to existing legislation and to strengthen those institutions responsible for the enforcement or implementation of the new procedures. This process of ‘Institution Building and Reform’ is crucial to ensuring that Turkey successfully transitions to the standards, norms, expectations and obligations of EU Member States.
Border management is included under the chapter of Freedom, Justice and Security of the Institution Building and Reform Process. This is one of the highest priorities of this process. When Turkey joins the EU, Turkey’s eastern borders will be Europe’s borders. Management of the eastern borders is complicated due to the geographical conditions in the region which creates the conditions for illegal border crossings. Landmines have been used to provide border security and prevent illegal border crossings, smuggling and security threats. These landmines now constitute a serious threat for border management.
The Anti-Personnel Mine Ban Convention (APMBC) was adopted on 18th September 1997. On 26th January 1998, the Chief of the Turkish General Staff issued a directive banning the use of anti- personnel mines by the Turkish armed forces. There have been no confirmed instances of the use of anti-personnel mines by Turkish forces since then. The Republic of Turkey acceded to the APMBC on 25th September 2003, becoming a State Party on 1st March 2004.
The Government of Turkey’s response to these developments included a request for assistance from UNDP. This focused on the demining of the eastern borders which are a military zone and expected to be so for the foreseeable future. Civilians are not permitted to live in these areas. In response to this request for assistance UNDP developed and implemented a project entitled “Technical Assistance for Socioeconomic Development through Demining and Increasing the Border Surveillance Capacity at the Eastern Borders of Turkey” in partnership with the Government of Turkey and the European Union. The project was designed to contribute to the prevention of illegal migration and cross-border crime on Turkey’s eastern borders by clearing the border regions of mines and providing effective humanitarian border surveillance tools for a technologically supported modern border surveillance system.
Review Objectives This review was designed to assess if the UNDP Turkey demining project in Turkey had realized the expected results as promised in the Project Document, while capturing the various interventions or activities that were initiated through the project.
The review was commissioned at the completion of phase one of the Project and was designed to cover all project activities from inception to the end of Phase One on 13 December 2017. To facilitate this process UNDP contracted an independent consultant, Mr Archie Law who has extensive mine action, development and humanitarian experience to conduct the review. Mr Law was most recently the founding Executive Director for ActionAid Australia and prior to that appointment worked as the Regional Mine Action Advisor for Africa for UNDP, the Policy Coordination Officer for UNMAS in New York and the Country Director for the Mines Advisory Group in Cambodia.
Page | 3
Methodology of the Review The consultant focused his efforts on collecting data concerning outcomes that contributed towards the achievement of the project’s objectives. The documents reviewed for the project review provided quantitative data and much of the review then focused on developing a qualitative analysis.
The members of the UNDP Technical Assistance Team (TAT) and UNDP Country Office staff members were interviewed by the consultant and a representative number of external stakeholders was identified for interviews. These included, senior officials from the Turkish Land Forces, Representatives of the Turkish Government, Donors, and service providers. Those selected for bilateral interviews and focused group discussions were largely selected by the UNDP Turkey TAT.
The consultant was in Turkey for 19 days and utilised the following methodology to fulfil his terms of reference:
1. A desk review of documents, such as the project document, project reports, technical documents, lessons learned documents, the International Mine Action Standards (IMAS) and other project related documents.
2. Interviews with partners, donors and government officials selected by UNDP.
3. Key informant interviews with knowledgeable individuals who worked with the project during its duration.
4. UNDP organised project visits to Igdir and interviews with UNDP staff and service providers.
The consultant thanks UNDP, and in particular the Project Manager and members of the TAT, for their generous hosting of the mission which is a credit to their professionalism, good will and ability to balance multiple priorities.
The Review Report The Project Review Report has four parts. The first part assesses the mine action context in Turkey and the second part assesses the strategic positioning, concept and design of the project. This part includes an analysis of the monitoring, evaluation and risk management approaches in the project. The third part of the report analyses the implementation of the project and its effectiveness and efficiency. The fourth part assesses the project’s progress and presents some recommendations for the future.
The final part of the report includes an assessment of the project’s progress against the core criteria of this review. The consultant notes that the review terms of reference requested an assessment of the application of a rights based approach and gender mainstreaming. There was little evidence that measures were taken to strengthen either of these methodologies in the project design and implementation. This is addressed in the recommendations section of the report. This is followed by a number of recommendations based on the evaluation objectives which hopefully assist with maximising impact and sustainability in the future.
The full Terms of references can be found at Annex One, together with the Methodology and Programme at Annex Two, a list of interviewees at Annex Three and a glossary at Annex Four.
Page | 4
1. MINE ACTION IN TURKEY Landmines and Border Management Turkey is contaminated with antipersonnel, antivehicle mines and Improvised Explosive Devices (IEDs). The majority of antipersonnel mines are along its borders, and were laid in 1955–1959. The 911km border with Syria is mined, and the mined area is on average 350 meters wide. The estimated size of the Syrian border minefields is 144.29km2. They are estimated to account for two-thirds of the mines and close to 90% of the remaining mined area in Turkey.
There are also mined areas on sections of the border with Armenia (1.10km2), Iran (21.33km2), Iraq (2.86km2), and Azerbaijan. Its western borders with Bulgaria and Greece, as well as the border with Georgia, are mine-free. A further 873 mined areas covering a total of 3.11km2 have been identified in “areas other than borders.” Mines were laid around military installations and eighteen of the 81 provinces in Turkey still contain mined areas. 1
Government forces used landmines during the 1984–1999 conflict with the Kurdistan Workers’ Party (Partiya Karkerên Kurdistan, PKK) in the southeast of the country. These mines have been progressively cleared since 1998. In addition to mines laid by Turkish security forces, non-state armed groups have used mines and IEDs which makes clearance processes more complex. The following table contains an estimate of the extent of the landmine problem on Turkey’s Eastern Border with Iran at the end of 2015 which shows that the contamination is most severe in Van, Acri and Igdir Provinces.
Table One: Contamination on Eastern Border with Iran (by Province) at End of 2015
SHAs CHAs Area CHA m2 AP Mines in CHA
Igdir 0 15 3,540,540 44,353
Acri 0 116 5,556,400 105,484
Van 30 109 12,180,500 41,704
Hakkari 8 227 52,371 15,657
2
The main reasons to lay landmines in these provinces differs from many other countries. Rather than laying mines during conflict in populated areas, the mines in Turkey were generally laid along the borders to prevent illegal migration, smuggling activities or to restrict PKK operations. These landmines are now limiting the operational capacity of Turkey in the border areas. They prevent the establishment of appropriate border surveillance systems and demining is now linked with border management reforms and ultimately Turkey’s accession to the European Union.
The Government of Turkey is making progress towards accession to the European Union and is following a National Programme for the Adoption of the Acquis as agreed with the European Union. The objective of the legal process is to implement the relevant amendments to existing legislation and to strengthen those institutions responsible for the enforcement or implementation of the new procedures. This process of ‘Institution Building and Reform’ is crucial if Turkey is to transition to the
1 Turkey, Mine Action, Landmine Monitor 2 Turkey Mine Action, Landmine Monitor, 2016
Page | 5 standards, norms, expectations and obligations of EU Member States. Border Management is evaluated under the chapter of Freedom, Justice and Security and is one of the greatest priorities of this process as Turkey’s borders will become Europe’s borders. Accordingly, Turkey is establishing a modern border surveillance system that uses state of the art technology and meets the EU targets for effective border management. Due to its geography Turkey stands at the crossroads between Europe, the Middle East and Asia. Turkey has difficulties in providing border security because of its lengthy mountainous land borders in the east and south-west and lengthy coastal borders in the south, north and west.
Illegal border crossings, mostly in the form of multinational organized crime (such as smuggling, human trafficking and terrorism) have created serious ramifications for Turkey and European border management. Illegal immigrants are smuggled into Turkey and Europe through Turkey’s land, sea or air borders or through security check points using counterfeit or stolen passports or concealed in cargo.
The numbers of illegal migrants crossing the borders between 2006 and 2009 were 15,270 and this number increased to 39,274 in 2009-2010. There were 44,415 irregular migrants apprehended in 2011, 47,510 in 2012 and 35,701 in 2013. More than half of these illegal crossings took place in the east and southeast of Turkey (Iran, Iraq and Syria) and 8,000 illegal crossings take place on the Iranian border each year. This is equivalent to 30% of the annual illegal crossings taking place in the project region. 3
Turkish Land Forces Command uses 4x4 vehicles, observation towers, border stations, thermal imaging cameras and barbed wire fencing to patrol the border. The minefields that stretch along the border prevent Turkish Land Forces from patrolling all areas along the border and prevent the establishment of a modern border surveillance system.
The mines are generally located approximately 500-700 metres behind the border and the patrol roads, fencing, border lightning and watchtowers are being built between the minefields and the border. Having the minefields 700 metres behind the border makes it impossible for patrol vehicles to access and patrol the border, particularly during the winter and in the mountainous areas. 4 Turkey is upgrading its border surveillance by renovating and reconstructing border stations through the Housing Development Administration of Turkey (TOKI), constructing and lighting roads at the borders, constructing telephone lines and constructing new fencing. Turkey is developing a Coastal Surveillance Radar System which consists of radars, towers, cameras and other electro-optic equipment. This will remove the need to staff many of the observation towers as this surveillance will be achieved with technology. When completed, the project will cover all of Turkey's coastal and maritime jurisdiction areas and this has been largely funded from Turkey’s national budget.
Turkey established a Task Force in 2002 which brought together the government agencies responsible for border management, asylum and migration. The task force included the General Staff, Ministry of the Interior (General Directorate for Public Security, Gendarmerie General Command and the Coast Guard Command), Ministry of Foreign Affairs, and the Under-Secretariat of Customs. They developed an overall strategy in 2002, under the coordination of the Ministry of Interior and established three separate working groups in the fields of “asylum”, “migration” and “external borders.” The Strategy Paper for the Protection of External Borders was issued on 14th April 2003 and based on the paper,
3 Technical Assistance for Socioeconomic Development through Demining and Increasing the Border Surveillance Capacity at the Eastern Borders of Turkey, UNDP Project Document 2014. 4 Ibid.
Page | 6 the “National Action Plan towards Implementation of Turkey’s Integrated Border Management Strategy” was approved by the Prime Minister of Turkey on 27th March 2006.
A working group on “external borders” continues to hold monthly meetings on the latest progress and problems regarding border surveillance and controls. A training strategy, programme and curriculum in EU standards for the new border police was established. As a result of the project “Development of a Training System for Border Police” and a “Common Manual of Checks at the External Borders” was developed for personnel working in the border areas.
International Engagement in Mine Action In 1996 the Ottawa Process was initiated under the leadership of Canada, Norway and a number of Non-Governmental Organizations (NGOs) under the umbrella of the International Campaign to Ban Landmines. The aim of the process was to end the suffering and casualties caused by anti-personnel mines around the world through a total ban on the use, stockpiling, production and transfer of anti- personnel mines. The Anti-Personnel Mine Ban Convention (APMBC) was adopted on 18th September 1997 and entered into force on 1st March 1999. Some 20 years later the convention is regarded as a major achievement of International Humanitarian Law (IHL). On 26th January 1998, the Chief of the Turkish General Staff issued a directive banning the use of antipersonnel mines by the Turkish armed forces. There have been no confirmed instances of use of antipersonnel mines by Turkish forces since. The Republic of Turkey acceded to the APMBC on 25th September 2003, becoming a state party to the Convention on 1st March 2004. Turkey has not enacted domestic implementation legislation, but maintains that its constitution, criminal code, and directives from Turkey’s Armed Forces General Staff, give legal effect to the treaty’s provisions. Turkey has been making steady progress to meeting its obligations under Article Five of the APMBC in a complex environment. A law on the “The Tender and Mine Clearance Activities along the land borders between the Republic of Turkey and Syria Arab Republic”, Law No. 5903 entered into force on 17th June 2009. The law tasks the Ministry of Defence to carry out demining activities on the Syrian border which have been funded through Turkey’s national budget.
This accelerated the coordination between Ministries. Turkey then developed plans to clear the minefields along the border with Iraq, Iran, Azerbaijan and Armenia. As a result of assessments conducted by the Government and Land Forces Command it was decided that it would be appropriate to use external assistance for the clearance activities in the eastern border regions.
Turkey requested an extension to the 2014 deadline for its obligation under Article 5 of the APMBC to clear all anti-personnel mines from mined areas under its jurisdiction or control and was granted an eight-year extension. This requires all mines from mined areas in Turkey to be cleared by 2022. Given the scope of mine contamination in Turkey it will be unlikely that the Government will be able to meet its Article Five obligation to clear all anti-personnel mines in mined areas under its jurisdiction or control by 2022 and therefore a further extension will be required.
Such extension requests can come under considerable pressure particularly from civil society advocates such as the International Campaign to Ban Landmines (ICBL) which robustly holds States Parties to account for the commitments they have made to the Convention. This pressure is expected to be more significant for Turkey’s second extension request and Turkey needs to use the next two years to prepare a rigorous and accurate extension request to create confidence in that request.
Page | 7
The most recent entry in the Landmine and Cluster Munition Monitor, which is an initiative providing research for the International Campaign to Ban Landmines (ICBL)-Cluster Munition Coalition (CMC), for Turkey from 23rd November 2016 contained a number of recommendations for Turkey which include some actions concerning this project. They include:
Turkey should move forward, without delay, with the clearance of non-border areas. TURMAC should provide additional details of ongoing survey of eastern border areas, and also provide information on how and when it will address the huge contamination in the eastern region that is not specified in its March 2015 workplan. TURMAC should re-consider its decision not to begin clearance on the Syrian border, where minefields under Turkish jurisdiction pose a risk to civilians fleeing fighting across the border. TURMAC should ensure application of best practice in land release, prioritizing technical survey over full clearance, to accurately determine the actual contaminated area. Turkey should report on plans for clearance of mined areas under its control in northern Cyprus, in order to meet all of its Mine Ban Treaty Article 5 obligations. Turkey and Cyprus should both heed the UN Secretary General’s call to facilitate access to all mined areas inside and outside the buffer zone to achieve a mine-free Cyprus.5
Turkey will be able to report in its Article Five Report to the next Meeting of States Parties that it has been moving ahead with the survey of the Eastern Borders area, has a plan to address the contamination in the eastern region, and has developed a land release process, The Operational Demining Process (ODP), which is making clearance efforts far more efficient. Such reports will contribute to building confidence in the efforts that Turkey is making to meet its international obligations.
The extension request to the States Parties to the Convention to extend Turkey’s deadline will need to be tabled at the Meeting of States Parties in early 2021 so that it can be reviewed and then discussed in 2022. There should be a draft extension request prepared by early 2020 so that the request can be reviewed and agreed on by all national stakeholders ahead of the Government of Turkey’s approval.
There is a need for a substantial investment in non-technical survey6 capability to facilitate the gathering of accurate data and the development of an accurate extension request. This data will then be entered in the Information Management System for Mine Action (IMSMA) which will require sufficient capacity in the newly formed Turkish Mine Action Centre (TURMAC) to conduct an analysis of the data before it forms the core of the extension request. The planning for this request needs to start as soon as possible and there will be a need for technical assistance to support TURMAC on this task. UNDP is well placed to provide that assistance.
5 Landmine and Cluster Munition Monitor 2017 6 IMAS defines Non-Technical Survey as “typically the starting point for the assessment of land, its categorisation as a suspected or confirmed hazardous area (SHA/CHA), and the associated processes of cancelling, reducing or clearing land for productive use. It involves a thorough investigation of new information about possible mine/ERW contamination, or a previously recorded hazardous area, generally without the use of mine action assets inside the suspected area”. United Nations, International Mine Action Standards, 8.10, NY, 2013
Page | 8
Land Mine Casualties in Turkey: People involved in criminal activities have a good understanding of the location of the minefields and appear to have largely avoided them. The exact locations of the mines have changed in some situations due to weather conditions or to the movements of soil. This does not seem as prevalent as presented by some sources and based on clearance statistics seems to have been most prevalent in a number of major earth slides originating from Mt Ararat. Mines may have also been moved for undetermined reasons over the last 40 years. They may have been moved by shepherds walking their sheep through the area or they may have been moved by smugglers, economic immigrants or terrorists. It’s impossible to account for these mines and the answer will never be known. The important action for Turkey to take is to clear all mines in known mined areas as per Turkey’s international obligations.
There are no comprehensive statistics concerning mine related casualties in the eastern border region where the UNDP project has been implemented. According to the Landmine and Cluster Munition Monitor analysis of 2008 media reports, which was largely compiled by the civil society organisation “Initiative for a Mine-Free Turkey” (IMFT), there were at least 100 new casualties due to mines, Explosive Remnants of War (ERW) and victim-activated IEDs, including 29 people killed and 71 injured. Of these reported casualties 72 were members of the security forces and 28 were civilians.7
The number of casualties in 2008 were similar to the casualties reported by the IMFT in 2007 with a further 101 casualties being reported including 28 people killed and 73 injured. Turkey reported 158 antipersonnel mine casualties in 2008 with 37 people reported as killed and 121 injured. It was not specified how many of the casualties were civilian, or if any civilians were “village guards” or members of the security forces. This total represented a significant decrease from the 257 casualties (53 killed and 204 injured) reported in 2007 and suggested a return to similar casualty numbers that were reported in 2006 (155 casualties; 24 killed and 131 injured). 8
Casualties continued to occur in 2009 with at least 48 new casualties of victim-activated devices (27 killed and 21 injured), reported by 29 June 2009. All but one of the casualties were male (the gender of one casualty was unknown) and the 47 male casualties included 42 men, one boy, and four casualties of unknown age.9
The total number of mine casualties in Turkey is unknown. The Landmine and Cluster Munition Monitor has reported that were 831 victim-activated mine/ERW casualties in Turkey between 1999 and 2008, including 250 killed and 581 injured. Turkey has reported that between 1993 and 2003 landmines caused 2,905 casualties, including 588 people killed and 2,317 injured. It was reported that more than 3,000 people (mostly civilians) have been killed and another 7,000 injured by mines along the Turkish-Syrian border since the 1950s. 10
According to Turkey’s latest Article 7 report to the Meeting of States Parties to the APMBC in 2008, 37 military personnel and civilians were killed and 121 were injured in 2008 by landmines used by the PKK/KADEK/Kongra Gel. The report does not differentiate between casualties caused by antipersonnel mines, anti-vehicle mines or IEDs, nor between victim-activated and command-detonated mines/IEDs.
7 UNDP: Country Specific Information for Justification of Demining Phase-I 8 Ibid. 9 Ibid. 10 Landmine and Cluster Munition Monitor 2016
Page | 9
In its Article 7 report for 2007, Turkey reported that 53 military personnel and civilians were killed and 204 injured by landmines used by the PKK/KADEK/Kongra Gel. According to the Annual Article 7 Report, for the reporting period of 01 January 2009 to 31 December 2009, Turkey has 1,702,982 anti-personnel mines stockpiled and the total quantity of mines in mined areas is 981,778. 11 The following table presents the data on mines and mine casualties based on Turkey’s Article 7 Reports.
Table Two: Mines and Mine Casualties in Turkey 2004-2009
2004 2005 2006 2007 2008 2009
Stockpiled 2.973.481 2.979.165 2.866.818 2.616.770 1.702.982 730.458 Anti- Personnel Mines
Mines in 920.552 984.313 983.166 982.777 981.778 979.417 mined areas
Retained for 16.000 15.150 15.150 15.125 15.125 Training
Deaths 25 39 24 53 37 25 Occurred
Wounded 123 155 131 204 121 58
According to a report prepared by IFDT the number of mines include 75,115 mines along the 42km border with Iraq, 191,428 mines along the border with Iran and 21,984 mines along the border with Armenia. 12 2. STRATEGIC POSITIONING, CONCEPT AND DESIGN The vision of the United Nations is a world free of the threat of landmines and ERW. There are fourteen UN department, agencies, programmes and funds that play a role in mine-action programs in 30 countries and three territories to achieve this vision. UNDP works in mine action in over 20 countries and manages several global partnership projects.
UNDP provides technical assistance and training, and it assists national and local authorities with a range of activities including but not limited to:
Establishing and sustaining national Mine Action Centres, to coordinate, prioritise, and assure the quality of mine action operations Establishing and launching information management systems Creating national legislative frameworks Developing resource mobilisation strategies
11 Article 7 Report to the Meeting of States Parties to the APMBC, The Government of Turkey, 2010. 12 Landmine and Cluster Munition Monitor
Page | 10
Project Planning The overall objective of the UNDP Turkey project has been to contribute to social and economic development through demining and the creation of more secure borders in Eastern Turkey. The specific objective of the project has been to contribute to the prevention of illegal migration and cross- border crime on Turkey’s eastern borders by clearing the border regions of mines and providing effective humanitarian border surveillance tools for a technologically supported modern border surveillance system.
UNDP Turkey was responsible for the implementation of the project and provided technical assistance for demining activities. This assistance was to include the following:
Project Management (Technical Assistance Team) Adjusting the National Mine Action Standards for the area covered Inception Activities Project quarterly and financial reports Accreditation of the bids during tendering Coordinating and contracting of clearance organizations Coordinating and contracting of Quality Assurance/control organizations Conducting a post-clearance review and report on the project.
As there was no National Mine Action Authority (NMAA) in Turkey when the project commenced, a Memorandum of Understanding (MOU) was signed on the 1st August 2013 between representatives of the Ministry of Interior (MoI), Turkish General Staff, Ministry of National Defence and Land Forces Command. This MOU agreed to ensure successful cooperation for mine clearance activities along the eastern borders of Turkey to take place.
The signatories to the MoU agreed to establish a “Project Implementation Committee” (PIC) that was to undertake duties such as managing and coordinating project activities, establishing cooperation between beneficiary institutions, following up on the project activities in cooperation with the contracting agency (UNDP) and project partners. The PIC for this project was intended to act as a de facto NMAA until a formal NMAA was established.
A NMAA is yet to be established and this has created challenges for UNDP and other stakeholders. This has required TURMAC to act as the authority for mine action as well as being responsible for mine action operations. The TURMAC reports to the Ministry for National Defence rather than to an inter- ministerial NMAA. This contains mine action within defence priorities rather than interacting with other Ministries such as the Ministry for Interior, Ministry for Immigration, Ministry for Health, and Ministry for Education that may have a stake in mine action. This creates a potential conflict of duty for TURMAC and it is important that an NMAA is established to enhance the good governance of mine action operations in Turkey.
The UNDP project document states that the purpose of the project was to clear known mined areas along the eastern borders of Turkey within a 24-month time frame. The document noted that the border regions Van, Agri, Igdir and Kars were all contaminated with anti-personnel mines, which have been surveyed, mapped, and marked with the minefield records held centrally by Land Forces Command. Due to the confidence in the accuracy of this information it was decided that survey capacity would not be required and the demining activities would need to include clearance, QA/QC (including post-clearance documentation) and the handover of cleared land.
Page | 11
UNDP established the TAT to be responsible for managing the clearance component, the quality assurance/quality control, certification component and for conducting a post clearance review upon completion of the project. The TAT consisted of key experts and administrative staff that were required to undertake this work and function in the Project Office (PO) that would be established within the facilities of the MoI. The team was to work in close cooperation with government partners in the PO and it was expected that the PO would develop into a Mine Action Centre (MAC) that would report directly to the NMAA when it was established.
The UNDP project document confirms that the most practical approach for conducting the clearance of the eastern border would be to split the project into three clearance areas/lots for contracting and one QA/QC/Certification contract covering all allocated areas/lots. The division of clearance lots would be based on geographical considerations, military controlled zones and terrain conditions and noted that contractors bidding on the clearance areas/lots may bid on single areas/lots or on all areas/lots.
The project document noted that by having a minimum number of contractors there should be a reduction in logistical and management costs and it would lead to better monitoring and control over the contractor. The contracting was to be done in accordance with UNDP’s rules and regulations, through competitive tendering using the UN Global Marketplace website.
Pre- Tender Planning Soon after ratifying the APMBC, the Land Forces carried out a review of known minefields. This was a basic review and focused on the compilation and copying of minefield laying records and installing safety fences around known minefields. This effectively increased the size of the hazardous areas and the review concluded that there were 225 minefields that needed to be cleared on the eastern border. There was an initial estimate that as the mines were laid to NATO standards and there were minefield maps the minefields could be cleared in two to three years.
There was an urgency from the Turkish Land Forces to “get this job done” which is understandable as this project was enabling improved border security for Turkey. This is a critical issue as it would be reducing the risk of “terrorists” entering the country as well as preventing illegal migration which is understandably a major concern for Turkey and Europe.
This urgency needs to be approached from Turkey’s perspective as the country is generously hosting 3,106,932 Syrian Refugees.13 This is the largest refugee population in the world and when combined with the long running struggle with the Kurdistan Workers Party (PKK), two major terrorist attacks and a failed coup d’état in 2016 it is easy to understand Turkey’s desire to clear the border minefields and enhance their border protection as soon as possible.
The desire to clear these minefields quickly, combined with perceptions that this was a relatively simple project has created unrealistic assumptions. Consequently, when the project proved to be far more complex than first envisaged and progress was slow, it created negative perceptions amongst many stakeholders.
There are no reports of a non-technical survey component being recommended. This concept is new to Turkey and the information from the Land Forces review formed the basis for the mine clearance plan of the Eastern Borders. The EU then engaged a consultant to assess the clearance plans.
13 Update: Durable Solutions for Syrian Refugees, UNHCR, 8th October 2017.
Page | 12
In April 2013, a UNDP Bureau for Policy and Programme Support (BPPS) mine action mission was deployed to Turkey. Its report warned against contracting a single clearance company, the implications of a low square metre price bid being accepted and the risk of starting work without a competent National Authority being in place. The mission did not question the scope, activities or accuracy of the Land Forces Review. 14
Monitoring, Evaluation and Risk Management UNDP then signed a grant agreement with the Government of Turkey in partnership with the EU to clear 225 minefields along the eastern borders. An assessment of some of the minefields was conducted by a UNDP technical consultant together with a Turkish military officer in August 2014. This assessment had very limited access to the minefields and the minefield plans were not made available to the consultant as the Government had classified them as restricted information.
The assessment provided superficial information concerning the terrain and the minefields and insufficient data to develop an accurate assessment of requirements. Consequently, the work plan that was developed to clear the 225 minefields was not based on an accurate desktop analysis or a comprehensive non-technical survey. The assumptions that these minefields were laid to NATO Standards, that the Turkish Military had all of the minefield maps which would be released to UNDP and that this was therefore a simple task were not questioned and there were no contingencies in the event of them being proven incorrect.
Given the difference in mine density from minefield to minefield and the wild variations in the size and features of minefields from flat ground to steep hilly terrain it is questionable whether this contracting methodology is appropriate for this project. Other methodologies such as contracting the clearance capacity to work to the priorities of a national demining plan that is shaped by TURMAC should be considered in the future. Contractors working to national priorities under the direction of the UNDP Operations Advisor, supported by QA to ensure the clearance work was done efficiently and effectively, would be more appropriate for this project.
This raises the question of outputs, outcomes and broader questions concerning the monitoring and evaluation of the project. Whilst the use of metres squared and mines cleared provide measures of outputs of the activities it provides little indication of the outcomes of the project. More work is required to build in outcomes to measure the success for the project. Some of the stakeholders interviewed for the review noted that the wall construction company had asked UNDP to clear land to enable three passageways to go through the wall. There were similar experiences where wall construction was enabled by the clearance project, where patrolling a wider area of the border is occurring and mine clearance is enabling the construction of electro-optical towers that will enhance border security.
This is all outcome related data and unfortunately this sort of information has not been sufficiently captured. There needs to be a shift to re-focus the results framework for the project on these types of outcomes. Working to a demining plan which is integrated with the priorities of the border management activities will certainly make it easier to determine the outcomes of the demining work.
Some key outcome focused questions could include; Can the border guards access previously inaccessible areas? How much time do they save per week and per month? Similarly, there could be an assessment of the length of time to access the border wall before mine clearance to enable access roads to be built and then after the wall has been constructed.
14 Lark, B., Eastern Borders of Turkey Minefield Clearance Project Overview and Lessons Learned 2015-2017
Page | 13
The Clearance Contract UNDP Turkey assumed the responsibility for the project operations which is one of the largest single mine clearance projects that has been implemented anywhere in the world. Whilst UNDP had the responsibility, it did not have the authority for project operations. This remained with the government which had no mine action arrangements in place such as legislation to enable the use, transport and storage of explosives.
The timing was unfortunate as there was no UNDP corporate technical assistance available for mine action. UNDP had decided to temporarily close its global mine action programme meaning that minimal technical support was available to UNDP Turkey as it prepared to manage its first mine clearance project. UNDP usually plays a role where it supports the capacity development of national institutions who are responsible for mine action activities. In Turkey UNDP was assuming an operational role where it was responsible for the clearance and quality management contracts.
UNDP Turkey approached this project as one where services would be provided through a competitive tender and UNDP would then manage the contractor. This approach is understandable in some ways as UNDP Turkey had not previously managed a mine action programme. There were some fundamental strategic gaps with this approach. For example, it is not clear how this project fitted in with the UN Development Assistance Framework for Turkey and how it linked in with Turkey’s commitment to the Millennium Development Goals (MDGs).
This approach did not include the establishment of a national mine action centre and possible UNDP support for the establishment of that centre which usually features in a UNDP supported mine action programme. Depending on the maturity of the programme and the requirements of the Government the UNDP support for many countries with a major landmine problem would include a UNDP Mine Action Support Team which includes a Chief Technical Advisor, Chief Operations Advisor, Quality Management Advisor and Information Management Advisor.
The direct management of a mine clearance programme, particularly during start up, requires experienced staff in the Country Office and the Project and is resource intensive. This was a new venture for UNDP Turkey who had not managed an RFP for demining activities before and there were no other similar UNDP programmes which could act as a source of expertise. This was also a new venture for the Government of Turkey who had been engaged in military demining but not in humanitarian demining.
There have been many examples of failed coordination in the field of mine action and the need for a national mine action governance structure, national policy and national strategy has been emphasised in many studies from the first seminal UN study on national mine capacities in 1997.15 The failure to recognise the need for a national mine action centre before mine clearance activities began meant that these arrangements were being developed whilst the clearance operation was being deployed. This led to many challenges. They included companies competing for the UNDP tendering and then deploying demining assets without having seen any minefield maps and national standards, including that for operator accreditation, which didn’t exist at that time.
The Clearance Tender The RFP for the three demining lots was primarily based on the number of minefields to be cleared. The selected contractor bid for all three of the demining LOTS that were specified in the RFP. The price of their three bids was less than the other contractors bid for a single LOT. Mechem have since
15 Eaton.R, Horwood.C, and Niland.N, The Development of Indigenous Mine Action Capacities, DHA, 1997
Page | 14 recognised this was a mistake on their part. The contractors were assessed as having similar technical capabilities and cost was the deciding factor. Mechem were then awarded all three LOTS despite advice not to pursue this course of action. Many others interviewed for this review have commented, with hindsight, that more than one contractor should have been selected to promote efficiencies in the project.
The former UNDP Chief Technical Advisor was critical of the RFP which explained that most mined areas have been surveyed and identified and do not require any subsequent survey process.16 The RFP should have required the physical checking of minefields and this would have required non- technical survey on the minefields being put out for tender.
The second RFP concerned the quality management of the clearance contractor and this was awarded to RPS Energy. Part of the role involved the training and deployment of Quality Assurance (QA) Inspectors which was highly ambitious given the training requirements and the need for an immediate deployment as the clearance contractor was working when they deployed.
Based on the workplan to clear the minefields the clearance contractor then submitted a plan which was a list of minefield numbers fitted into two demining seasons. These demining seasons had a duration of eight months per year as a result of the need to stand down the clearance teams during the severe winter months. UNDP hoped that technical survey17 activities could be applied when the clearance activities began. There were also hopes that a land release approach could be applied although there was no standard for such an approach in Turkey.
There were then immediate delays as there was no accreditation process, no clearance standards, and no agreed tasking in Turkey. This and the need to conduct technical survey meant that little clearance work occurred in year one of the project. This is an expensive exercise when clearance assets are not involved in clearance activities.
There were additional delays due to other requirements such as authorising and constructing Mine Detection Dog (MDD) areas, agreement on the provision, carriage and storage of explosives and the location of a Central Demolition Site. This is all highly sensitive work and required much work from the UNDP mine action team.
The problems with sequencing the project activities created challenges and the aftermath is still being experienced today. Numerous stakeholders interviewed for this review have voiced concerns about the professionalism of Mechem. These range from problems with Mechem’s relationship with the Quality Management Contractor, to TURMAC, to UNDP and the Turkish Land Forces. Whilst there have been a number of challenging relationships with Mechem and other stakeholders it must be acknowledged that Mechem has also delivered the clearance results that the programme is now celebrating. These relationships need to be repaired and strengthened if Mechem is to continue its work on the project in 2018.
Some have mentioned that UNDP Turkey was providing task orders to Mechem although UNDP did not have the permanent presence in the minefields to ensure that Mechem were fulfilling the
16 Lark, B., Eastern Borders of Turkey Minefield Clearance Project Overview and Lessons Learned 2015-2017 (Draft) 17 IMAS defines Technical Survey as “A physical intrusive process into a hazardous area which may provide enough information to allow an informed assessment of the clearance needs. This process is known as a technical survey and although it may be a separate activity, it is often integrated with clearance and can be undertaken before, during and even after clearance”. United Nations, International Mine Action Standards, 8.22, NY 2009.
Page | 15 requirements of those tasks. This created a situation where Mechem was perceived to be making their own decisions about which task they do and how they do it.
Whilst the situation has improved, UNDP still has a light capacity in the field to manage a project this size. This contract is currently being managed by one part-time expatriate staff member and two full- time national staff members. One national staff member has been forced to take leave from the project for personal reasons which has led to his responsibilities being assumed by his colleague. His role needs to be backfilled as there is not a sufficient staff presence to carry staff vacancies in a project this size. On a project this size there should be at least one full time international staff member responsible for Operations, and the two national staff field coordinators. The field team should be reinforced before the 2018 demining season begins.
The delays resulting from the rush to get the project moving due to a poor planning process created a reputational risk for UNDP Turkey. These risks include the perception that the UNDP Turkey is not able to successfully implement large projects, that the organisation has to defer to UNDP New York on seemingly minor issues, and that UNDP has complex and cumbersome procedures. These are perceptions that appear to be embedded with some stakeholders and UNDP should be reviewing and enhancing its approach to reputational risk to respond to these perceptions. 3. IMPLEMENTATION The Establishment of TURMAC During the preparations of the Project, the TURMAC was established in February 2015 with the “Law No: 6586 on Establishment of National Mine Action Centre and Amendments of Various Laws”. TURMAC is mandated to execute the actions to clear mines and/or unexploded ordnance for humanitarian purposes within the borders of the Republic of Turkey. The formal hand over of the Project from Ministry of Interior to TURMAC was done in November 2015 with a protocol signed between the parties.
Since the TURMAC was established during the preparations of the Project in February 2015, the management arrangements of the Project were revised and TURMAC assumed the responsibility for the execution of Component I of the Project (Component 1: Clearance of landmines in the eastern border regions of Turkey -Direct Grant Contract). In that respect, the management structure of the Project was revised ensuring that a Senior Project Officer or SPO, appointed by TURMAC and MoI will be responsible for the overall execution of the Project.
During 2016 and 2017, a level of uncertainly existed within military intuitions, and TURMAC like every other institution in the military have been affected. The speed of rotation in TURMAC, for example senior staff members can be in their roles for a matter of months, has a negative impact on capacity development as well as on demining operations. The high staff turnover needs to end to enable institutional capacity to be put in place and for critical systems for information management, operations and quality management to be put in place.
It is encouraging that the Government has approved the structure for TURMAC as recommended by TURMAC and UNDP. However, it is estimated that TURMAC is still working at 50% of the required capacity. There is a goal of recruiting civilians to 85% of TURMAC posts in the future. This corresponds with the Government’s aim of making the Ministry of National Defence more of a civilian led organisation as it is in many other countries in Europe.
Since the operations of demining and the establishment of TURMAC coincided, the capacity development activities become an urgent and vital requirement to ensure the successful
Page | 16 implementation of project activities, clearance operations and to reinforce decision-making processes. UNDP has a focus on supporting governments and the organisation has a rich history of success in this field. As there was no mine action centre in Turkey prior to the establishment of TURMAC one would expect that the initial focus would have been to build a robust mine action centre and this would be expected to take two years in Turkey. In the rush to begin the clearance work the establishment of a national capacity was temporarily neglected.
UNDP rectified this situation and deployed an experienced and capable staff member to work on the capacity development of TURMAC in 2016. However, there were no supporting staff assigned to the role and the weight of the project was on the clearance contracting rather than developing an integrated project which focused on developing a national capacity to clear mines. The approach to developing capabilities in TURMAC has been to create an enabling environment through a clear legal status, strategic framework, and organizational structure.
Thanks to the efforts of UNDP and TURMAC there is now a clear mandate from the Government of Turkey under Law 6586 and the organisational structure is expected to be approved soon. Additionally, the development of priority National Mine Action Standards (NMAS) with a formal system in place to review and update the current set, as well as new standards, is required.
TURMAC requires further support. There is a pressing need for UNDP to complement its management of the clearance and QA/QC contract with a greater focus on capacity development. UNDP has worked with TURMAC to develop a capacity assessment and this should form the basis of a long- term capacity development plan. At present one UNDP staff member is responsible for capacity development. There needs to be a transition so that in 2019 all UNDP mine action specialists have a shared responsibility for capacity development in their job descriptions. This capacity can be used to build internal systems and structures as well as improving the understanding and knowledge of the TURMAC staff.
There have been a number of impressive capacity development achievements for which UNDP and TURMAC are to be congratulated. A national strategy has been developed, priority national standards have been completed, an innovative national land release policy, referred to as the operational demining process, has been developed and there have been many trainings and workshops. These include an introduction to mine action as well as introductory and more advanced trainings on IMSMA and quality management.
Mine Clearance By June 2016 it was clear that it would be impossible to clear the 225 minefields in the 384 operational days in the contract. To the credit of all involved some critical decisions were taken and things began to turn around. UNDP made an important decision to conduct a reconnaissance of the 225 minefields and importantly requested that a senior staff member of the contracted company participate in this task. This reconnaissance showed that based on the characteristics of the minefields, some of which were on steep hills and slopes, the clearance of the mined areas would take at least five years more than planned. These findings also depended on an agreed approach to land release being agreed to before the 2017 demining season commenced. This led to UNDP, the European Union and the Government of Turkey working together and developing a new and phased approach to the implementation of the project which proved to be the “game changer” that everyone was searching for.
Page | 17
Phase One The joint site- reconnaissance of UNDP and TURMAC in August 2016 proved that the difficulties involved in identifying mine rows, and accessing and physically clearing many of the minefields were critically underestimated during the pre-contracting phase due to the lack a comprehensive survey.
The Steering Committee agreed that the previous clearance targets were ambitious and unachievable within the budget and the duration of the contract. The Steering Committee then agreed to reset the clearance targets to reflect the outputs of site reconnaissance and the results of the clearance operations in 2016.
A realistic work plan based on the introduction of the Operational Demining Process (ODP) which included a list of 20 minefields (3,570,710 square metres) in Iğdır containing approximately 44,537 mines was approved for the 2017 demining season. The clearance of these 20 minefields would mean that Iğdır province would be completed and free of known minefields.
Table Three: Phase One Targets
Total sqm Number of Number of Mines MTRs
Phase I 3,570,710 sqm 44,537 over 20 MTRs over over 190,023 mines 225 (8%) 11,700,907 23.5% sqm (30%)
Phase Two The Phase 2 minefield survey, which was conducted by a group of TURMAC and UNDP experts, found that security in southern Van and Hakkari would not allow humanitarian mine clearance and a full non-technical and some technical survey work would be required before any mine clearance began.
In addition, the results of the joint site-reconnaissance, which was conducted in August 2016 in Phase I Project areas (Iğdır, Kars, Ağrı, Van) showed that the difficulties involved in identifying mine rows, accessing and physically clearing many of the minefields were critically underestimated during the pre-contracting phase due to the lack a comprehensive survey.
The Steering Committee agreed during the October 2016 meeting that the Phase 2 mine clearance areas could be redefined to cover the areas that were not covered in the redesigned Phase I Project. Based on the reconnaissance in August 2016 under Phase I of the Project and the minefield survey conducted ahead of the design of the Phase II project, the number of minefields to be cleared in Phase 2 was agreed to be 66 minefields in Ağrı Doğubeyazıt. This review notes that this is a challenging and ambitious target and the contractor will be working in more difficult terrain during 2018.
Page | 18
Table Four: Phase Two Targets
Total sqm Number of Mines Number of MTRs
Phase II 2,818,100 sqm 49263 over 190,023 mines 66 MTRs over 225 (30%) for 66 MTRs 26 % over 11,700,907 sqm (24%) (Annex I)
[Plus 769,202 sqm as a
contingency area]
The introduction of these new phases has successfully established a foundation for the success of the project and there have been some impressive results. For example, the removal of 25,000 mines in 2017 as illustrated in Table Five below is a substantial reduction of the risk of people and livestock detonating mines and either dying or suffering life threatening injuries.
Table Five: Mines Found as of 7 December 2017
MINES NO MTR NO CONTRACT FOUND PERCENTAGE (%) MTR CONDITION DM 11 M14 M2 DM 11 M14 M2 DM 11 M14 M2 1 114 237 0 0 235 0 0 0.99 0.00 0.00 HANDOVER COMPLETED 2 40 2050 0 127 2041 0 127 1.00 0.00 1.00 HANDOVER COMPLETED 3 39 1464 0 0 1140 0 0 0.78 0.00 0.00 IN TURMAC 4 33 3555 0 48 3327 0 48 0.94 0.00 1.00 HANDOVER COMPLETED 5 38 8043 0 6649 0 54 0.83 0.00 0.00 IN TURMAC 6 37 4798 0 83 4722 0 50 0.98 0.00 0.60 IN TURMAC 7 36 9650 0 114 3998 0 25 0.41 0.00 0.22 IN TURMAC 8 35 1244 0 23 1066 0 24 0.86 0.00 0.00 IN TURMAC 9 41 196 0 49 149 0 27 0.76 0.00 0.55 IN TURMAC 10 31 1956 0 40 1917 0 34 0.00 0.00 0.00 IN TURMAC 11 506 75 0 41 22 0 7 0.29 0.00 0.17 IN TURMAC 12 129 0 5 0 0 1 0 0.00 0.20 0.00 SUSPENDED 13 130 0 4 0 0 1 0 0.00 0.25 0.00 SUSPENDED 14 131 0 3 0 0 1 0 0.00 0.33 0.00 SUSPENDED 15 132 0 2 0 0 1 0 0.00 0.50 0.00 NOT STARTED 16 133 0 2 0 0 0 0 0.00 0.00 0.00 SUSPENDED
38 CLUSTER 27426 16 350 17746 0 163 0.65 0.47 TOTAL FOR 16 MINE 33268 16 525 23349 4 338 0.70 0.25 0.64 FIELD MINES TO BE 9919 12 187 FOUND
Page | 19
Contracting Changes As a result of the above changes and a reduction in the number of minefields to be cleared there were implications for the contract with Mechem. The conditions of the contract specified that the contractor was to paid on a quarterly basis for the minefields that had been handed over to TURMAC.
There were many discussions to clarify the definition of “a fully cleared minefield”. For example, some discussions focused on whether there needed to be a mine free area or was the aim to be the collection of all mines which corresponded to TURMAC’s minefield records? As a result of this negotiation it was not possible to pay the contractor nor was it possible for UNDP to recoup the advance of USD 3.9 million that had been transferred to the contractor at the commencement of the contract. This resulted in strikes, security problems, operational delays, and reputational damage for all parties.
Contract re-negotiations commenced with Mechem in 2016 and the payment modality was amended. Payment was now to be made on the number of mines cleared with a secondary payment for work on Suspected Hazardous Areas (SHA’s) and a final payment on handover. This resolved the cash flow problem and was appropriate for minefields in phase one where the contractor was working in high density minefields.
The revised work plan and accompanying contract amendment led to a ceiling price on the clearance contract of USD13,416,501 with USD7,498,491 being allocated for phase one and $5,918,010 being allocated for phase two.18 This is approximately USD6,500,000 less than the contract Mechem won when it was initially tendered.
The revised contract between UNDP and Mechem was signed in May 2017 which unfortunately resulted in a two-month delay in demining operations. This reduced the available demining time in 2017 from eight months to six months and this delay is partially responsible for the failure to meet the requirements of the 2017 work plan.
Reviewing the Contracting Model The contracting model needs to be reviewed when planning the next phase of the project from 2019 onwards. The model currently focuses on the number of mines collected and the handover of minefields. A more efficient approach can be developed.
The current system requires the clearance contractor to clear minefields and to provide technical survey as part of the clearance capacity. However, this has been contested and there were disagreements concerning Mechem’s obligations to provide technical survey. A number of people interviewed for this review noted that the Mechem mechanical assets could have been deployed for technical survey which would have increased the efficiency of the clearance process and this needs to be clarified for the 2018 demining season and beyond.
There are currently a number of different views amongst the UNDP TAT on how technical survey and clearance should complement each other. From the perspective of this review the technical survey capability should remain embedded in the clearance capability. Technical survey should be done within a clearance task as opposed to having a stand-alone technical survey capability that can be used to reduce the size of minefields for clearance. This is an area for the TAT to build a shared understanding of the desired approach to technical survey and clearance as the variety of views in the team is creating a lack of clarity.
18 UNDP Contract Agreement with Mechem
Page | 20
The Operational Demining Process The Operational Demining Process is based on the land release approach contained in the International Mine Action Standards (IMAS).19 It’s adoption has been one of the major successes of the project. The ODP was developed by UNDP with the support of the newly formed Operational Working Group (OWG) which involved TURMAC, UNDP and demining contractors and met every second week in the field. The group is responsible for managing task planning, task execution and any other outstanding task related matters.
This group was instrumental in making a big breakthrough for mine action activities in Turkey and developing a process to ensure that the minefields could be handed over when completed. This process clarified land release approaches which did not require every square metre of the minefields and the land in the safety fences erected after the signature of the APMBC to be cleared. An agreement on the process was reached in February 2017 prior to the beginning of the 2017 demining season and this was a major step forward for the programme.
The ODP was developed to provide clear operational guidelines to contribute to the overall quality and safety management of operations. The process consists of:
1. Targeted demining (Clearance and/or Technical Survey)
2. Extended search A (Search for missing mine/mines)
3. Extended search B (QA / confidence building)
4. Handover (Task completion)
During the contracting phase the initial information indicated the approximate location of the Suspected Hazardous Area (SHA) of the Project with no clear borders, GIS information or task definition. This information does not define the Confirmed Hazardous Area (CHA) which is required for efficient demining operations.
Based on the SHA targeted demining operations (which comprise clearance and/or technical survey) are then conducted to confirm the location of the CHA. An amended task plan is then developed and follow-up activities are applied to locate missing mines. Following this search, the clearance contractor is required to conduct additional sampling of not more than 5% of the area around the minefield that hasn’t been cleared to increase confidence that all of the mines have been cleared.
The clearance contractor then provides a report on the completed task which includes a presentation of the cleared areas and what clearance assets and methods have been used as well as the GIS and additional data required. The remaining area of the contract is then reported as a cancelled area in the cancellation report.
This process can be fine-tuned and some interesting suggestions emerged during the interviews for a more streamlined ODP to speed up clearance efforts. For example, when a clearance team is deployed there could be an initial technical survey by the clearance team to identify the mine rows. The
19 IMAS defines the Land release process as establishing and improving the definition of where mines/ERW are to be found (and where they are not) through the application of all reasonable effort, until it can be shown with justifiable confidence that mines/ERW are either not present in an area or, if they were found to be present, have all been destroyed or removed from that area. United Nations, International Mine Action Standards 07.11, NY, March 2013.
Page | 21 clearance team could then clear these mine rows and technical survey could then be conducted by dogs or machines to provide additional confidence that the remainder of the minefield is free of mines. It would be useful for the UNDP TAT to continue discussing these ideas and creating pilots to assess their applicability.
The hard work of all partners in the project to establish TURMAC, deploy contractors, and develop processes such as the ODP was rewarded in 2017. Standing up an operational mine action programme in less than two years, with minimal capacity, in a complex environment is indeed a remarkable achievement. An achievement that all who have been involved, and those who are currently involved with the project should be justifiably proud of.
The Consultant was interviewing the UNDP Field Coordinator during the early stages of his mission and that conversation was a small window into the complexity of the field operations. The Field Coordinator referred to a list of approximately 10 names on his mobile phone and explained that all of these numbers which included Land Forces, the Turkish and Iranian Border Guards, local administrators and other stakeholders needed to be notified when the demolitions of mines were proceeding. These demolitions usually occurred at the end of the week. He explained that he had to call and speak to some stakeholders in advance as he knew who would be checking their phones and who wouldn’t. It is this sort of commitment and finely tuned sense of partnership that has driven the project and the staff are to be congratulated for driving this progress.
Page | 22
4. CONCLUSIONS AND RECOMMENDATIONS The main finding of this review is that the project has achieved most of what it set out to do in spite of a very challenging beginning. UNDP and its partners should take pride in this result. This is a story of perseverance, innovation and learning and the following table rates the project against the UNDP evaluation criteria:
Project Assessment CRITERIA RANKING 1.Relevance Satisfactory (S) The project is a vital component of the Government of Turkey’s strategy to enhance their border security and the Government is committed to clearing the border minefields. The demining of land along the border is an integral component of the project that establishes modern border management practices which will enable Turkey to respond to the challenges that include unauthorised migration, terrorism, people trafficking and smuggling. 2.Effectiveness Moderately Satisfactory (MS)
The project initially aimed to clear 225 minefields in two years and this was not achieved. The game changing decision to divide the project into two phases established a more appropriate direction for the project. The project cleared over 90% of the phase one minefields which is a substantial achievement given the difficulties that were faced at the beginning of the project. There were also substantial achievements in developing the capacity of TURMAC. 3.Efficiency Satisfactory (S)
The project, including the capacity development objective, was implemented in an efficient manner. The design focused the contracts on clearing the minefields on the Border and new processes such as the ODP greatly enhanced clearance efficiencies. 4.Impact/Results Satisfactory (S)
The project delivered outputs that included the clearance of 25,000 landmines and the clearance of 11 minefields. These outputs led to some short-term outcomes that included improved mobility for border patrols, road construction and the construction of border towers and fencing. The work plan could have prioritised the clearance of the minefields to respond to government priorities for the border management project. 5. Sustainability Moderately Satisfactory (MS) 5.1 Financial Resources Unlikely (U) The project relies on funding from the European Union and the Government of Turkey to continue beyond 2018 assuming that a request for an extension to cover activities in 2018 is approved by the EU. The prospect of the EU providing funding for the project beyond 2018 is highly uncertain and other than the Government, here are no other donors in the pipeline as of December 2017. 5.2 Socio – Political Likely (L) Strengthening border management is a major political, economic and social issue in Turkey and the stakeholders are highly engaged in the project and committed to its success. The rapid establishment and support for the TURMAC is an indicator of this commitment. 5.3 Institutional Framework and Governance Likely (L) A number of pieces of legislation as referred to in this review have been passed to promote the implementation of this project. The TURMAC has been established with the appropriate policies and governance structures. The technical know how to run the project remains under development.
Page | 23
Conclusions and Recommended Actions After the success of 2017 there is good reason for optimism as the demining project is in a good position to enjoy its first uninterrupted demining season in 2018. UNDP has contributed much to this positive forecast through astute contract management, capacity development and partnership and coordination with the stakeholders of the project.
It’s important that incremental changes are made to the project in 2018 and the emphasis must be on meeting UNDP’s contractual obligations to the donor which include the capacity development of TURMAC. The development of a national capacity to manage mine action activities should be one of the primary legacies of this project.
The following conclusions and recommendations for 2018 are based on the assumption that the donor approves the project amendment for 2018. The conclusions and recommendations are shaped around the key evaluation objectives which have guided this review. They include:
Strategic Positioning, Concept and Design Implementation Partnership and Coordination Monitoring, Evaluation and Risk Management Rights-based Approach and Gender Mainstreaming
Strategic Positioning, Concept and Design There is a tension between the desire of Turkey to quickly clear mined areas to enhance its border surveillance capability and UNDP’s mine clearance contracting that conforms to national standards which are based on the International Mine Action Standards. These tensions were not resolved before the project was designed, an RFP developed and mine clearance assets contracted. This led to challenges for all partners. To the credit of all involved in the project there was a re-set of the project into two phases and realistic and achievable objectives, outputs, activities and inputs were established for 2017. More ambitious targets were then established for 2018 and it will be a challenge to achieve these targets.
Most partners understand that this project will potentially run for 10-12 years although this must be determined by non-technical survey options which will verify the correct planning period.
If this phased approach is adopted it needs to be implemented in such a way that it is consistent with the National Mine Action Strategy which has been approved by the Ministry of National Defence. This strategy focuses on three strategic objectives:
1. To perform mine action in accordance with pre-determined priorities which include creating policies on the surveillance of mines/explosives, clearance activities, mine risk training, aids for mine victims, destruction of mines kept in stocks and to plan, manage and perform these activities in accordance with the pre-determined priorities. 2. To enhance Turkey’s border security and establish professional border management which will prevent illegal migration, smuggling and terrorist acts on the border between Turkey and Syria. 3. To develop a sustainable national capacity by building the staff capability and institutional capacity to plan, manage, coordinate and implement the responsibilities of Turkey to the APMBC.
Page | 24
This phased approach would lead to blocks of three-year planning exercises which should be based on the principles of human rights and protection. This multi-year project should support Turkey to meet its obligations to the APMBC, strengthen its border management capability and be integrated with Turkey’s commitment to the Sustainable Development Goals (SDGs).
This approach is similar to the methodology that the Government of the United Kingdom is using to clear the minefields on the Falkland Islands and some of the lessons from the Falklands could be used in this project. This approach will build more confidence with stakeholders such as the donor community and the Government of Turkey and will presumably lead to tranches of funds being programmed in response to the developments on the ground.
Implementation The implementation of the project has been generally successful and many activities have been achieved as explained in this report. This is a credit to the leadership and management of the project.
Future planning for the project in 2019 and beyond needs to focus UNDP’s work more on building an effective capacity development programme which involves UNDP technical staff. It is suggested that UNDP consider the following approach:
1.1 Advocate for the establishment of a NMAA to have overall responsibility for mine action in Turkey. There is a need for a ministerial level awareness and understanding of Turkey’s international obligations in the field of mine action which can only be achieved through the creation of the Authority. The NMAA will create organisational clarity for the TURMAC on its operaitonal roles and responsibilities and will help to create a more effective TURMAC and a more effective national mine action programme.
1.2 Continue and increase the capacity development of the TURMAC and enhance the ability of TURMAC to coordinate all mine action operations in Turkey whether they be conducted by the Turkish Land Forces, the Gendamarie, commercial operators or NGOs.
UNDP is uniquely positioned to provide this support to the Government of Turkey and this is a low cost and potentially high return activitity for UNDP to engage in. This would require UNDP to build a Mıne Action Team that provides advisory services across the core activities of a Mine Action Centre. This support can be used to provide guidance to Turkey as it prepares its extension request on its article five obligations under the APMBC which is a major piece of work and a priority for the future of mine action in Turkey.
1.3 Work with TURMAC to build its capacity to conduct non-technical survey activities which will be a longer term initiative. This will have resource implications and would require an increased investment from UNDP to hire a technical advisor for survey activities and the recruitment and training of TURMAC staff who have the appropriate competencies and experience to quickly understand their responsibilities.
1.4 Continue the current UNDP project into phase three with a new RFP based on surveyed minefields that should be conducted by non-technical survey experts. The results of the survey would form the basis for the development of a work plan for phase three which could
Page | 25
be planned over three years. The nature of the workplan would then determine what assets are required and tendered for through the RFP.
UNDP could be responsible for the contract management and TURMAC would be responsible for the operational management of the project with capacity development support from UNDP. This would require survey activities to begin early in 2018 to enable the development of a project document and the identification of a donor to partner on this project from the beginning of 2019.
Partnership and Coordination UNDP has been a catalyst for collaboration and partnership and has successfully mobilised national actors and international donors around the demining project. A number of those interviewed for this review have commented favourably on UNDP’s coordination capabilities and this respect for UNDP is evident with Steering Committee members. In addition, UNDP’s support for TURMAC has provided a focus for coordination efforts in the field of mine action in Turkey.
UNDP was instrumental in working with Turkish Land Forces and the European Union to develop an assessment of the requirements of this project. Whilst this assessment was insufficient, UNDP successfully deployed the clearance and quality management components and the capacity development services which have delivered many of the projects successes. UNDP also worked successfully with the European Union and TURMAC to successfully reshape the project into two phases of activities. This was a major reason for the success of the project throughout 2017.
The emergence of TURMAC is also a major success for UNDP and indicative of how UNDP has supported the Government and the EU to integrate TURMAC and capacity development support into this project. This success is visible with the development of the national standards, the introduction of IMSMA and the mentoring and support for TURMAC to build an understanding of humanitarian mine action. UNDP also developed the land release methodology “the Operational Demining Process” and was then able to guide and support TURMAC and other members of the Steering Committee to build their understanding of this methodology. This is one of the major successes of this project.
This review is recommending that UNDP increase its support for TURMAC as the organisation’s development will be one of the legacies of the project. The capacity development of national institutions is one of UNDP’s core activities and UNDP is encouraged to work with TURMAC and partners to accelerate its capacity development efforts based on the recent capacity assessment. These efforts should focus on building the institutional capability of TURMAC to manage operations/quality management and information management which are the core of a mine action centre’s responsibilities.
It would be useful for the TURMAC operations, quality management and information management staff to be re-located to the operational sites as a capacity development initiative. Whilst there has been substantial progress in building an increased capability in these three core areas, future improvements can only really occur with the benefit of coaching and mentoring from experienced professionals.
It is recommended that the TURMAC staff work with the UNDP Chief of Operations to build their understanding of operations work through on the job training. Similarly, there should be an assessment of whether the TURMAC quality management staff can be attached to the RPS quality management staff to accelerate their learning.
Page | 26
Monitoring, Evaluation and Risk Management The project has successfully captured data and monitored the clearance of minefields in metres cleared and the number of mines cleared. It has also been proven to be nimble in making some major decisions to build lessons learned into the project such as the development of the Operational Demining Process. However, there is more work required to build the outcomes of success into project design and planning. This could include UNDP clearing land for broader national priorities such as passageways through the wall, the construction of towers and fencing construction, and enabling patrolling of the areas around the wall.
Such an approach would be easier if the work planning process involved the clearance capability being appropriated to the priorities of the border surveillance project as opposed to clearing the minefields which is currently the operating model. Such a planning process would make it easier for the TURMAC to record outcome level data in IMSMA and would lead to a shift in the focus of mine action in Turkey.
There was no evidence of an evaluation plan being built into the project. It would be useful for this to happen in advance of the project’s completion in 2018. Such a plan would use the monitoring data from the project to work out the big questions that need to be answered, the best way for them to be answered, and determine who can answer them. This can feed into the methodology and programme for the next review in 2018 which should build on the methodology and programme used for this review. This will help ensure that all monitoring and information management is being directed to the review and will provide a more accurate assessment of the project’s success and lessons learned.
There could be additional work in using the risk management framework to assess and mitigate the risks that face the project. At present risk management is largely practiced through the clearance contract although this should be a theme that integrates all of the project activities including capacity development. The UNDP TAT could consider updating the risk management on a monthly basis and then having the items which are high risk and high impact on the agenda of regular meetings so that they can be closely monitored.
Rights Based Approach and Gender Mainstreaming There was not a rights-based approach integrated in the project and this is an area which requires further thought from the Steering Committee and the Project staff. There are opportunities for rights based approaches to be used in the border management strategies particularly in Turkey’s response to the challenges posed by issues such as mass migration and people trafficking. The demining project and the border management project presents UNDP with an opportunity to work with the Government of Turkey and Land Forces to formulate approaches that promote the protection of people’s rights, particularly those who are fleeing persecution, conflict and the survivors of trafficking.
A rights-based approach would consider how the voices, agency and immediate needs of those arriving at the border are integrated into Turkey’s immigration policy and operational response. This should include a focus on the access to food, water, shelter and medical services on arrival at the Border. This would potentially be an opportunity for UNDP to work more on gender mainstreaming activities such as ensuring that the survivors of sexual and gender based violence receive access to services, that women can access sexual and reproductive health services and that women’s rights are protected. Such an initiative would enable UNDP to respond to some gaps in how the project involves women in its work.
The consultant did not witness any gender mainstreaming activities in the project nor was the issue of gender and mine action raised in any of the interviews with stakeholders. Some possible steps
Page | 27 that could be taken in future is requiring contractors to employ and train women as deminers and support staff and there should be a target established, for example 50/50, in hiring women for demining work. All evidence from many different countries indicate that women are highly capable deminers and the project needs to prioritise and act on this. UNDP could support TURMAC to work with stakeholders and develop gender in mine action guidelines based on the international guidelines which would apply to TURMAC, UNDP mine action and all mine action operators.
Page | 28
co Turkey
ANNEX ONE
Terms of Reference (TOR)
International Consultant for the Post-Project Review of the Socioeconomic Development through Demining and Increasing the Border Surveillance Capacity at the Eastern Borders of Turkey Phase-I
Location: Ankara and lgdlr (Duty Station) Type of Contract: Individual Contract Post level: International Consultant
Language required: English Duration of the Contract: November 2017 December 2017
I Background and Context Border management is regarded as being one of the most priority areas of EU Accession process for Turkey. After the membership to the EU, as Turkey's eastern borders will be the external borders of the Union, management of a comprehensive border security and its implementation constitutes an important subject. However, management of the eastern borders is not an easy task due to geographical and climate conditions of Turkey in that region. Moreover, this region constitutes a big problem in terms of being the illegal crossing route and in order to provide the border security in this region, landmines had been used mostly to prevent illegal border crossings, smugglings and resulting security challenges. For the time being these land mines constitute a very serious threat for border management in that region,
The landmines are generally located in border areas mainly in Armenian, Nakhcevan, Iranian, Iraq and Syrian Borders. This constitutes a very important problem in terns or providing an efficient border management system. These land mines arc generally laid along the borders to prevent illegal movements or smuggling activities or for security challenges. Under the current circumstances, in addition to the humanitarian concerns, these landmines limit the operational capacity of Turkey in the border areas. They prevent the establishment of good functioning border surveillance systems and as a result lead to a decrease in the efficiency in the fight against illegal movements, cross-border crimes and smuggling. For these reasons, demining is evaluated as an
Page | 29 indispensable part of the border management reforms. Turkey targets to have these landmines cleared to provide a more secure way of protecting the borders by means of technologically supported border surveillance systems and achieve the targets set out by European Union for a good functioning border management system.
Within this framework, having an ultimate objective of contributing to the social and economic development through demining and more secure borders in Eastern Turkey, the "Socioeconomic Development through Demining and Increasing the Border Surveillance Capacity at the Eastern Borders of Turkey Phase I and Il" was developed. In first phase of the Project, it has been planned to clear the mined areas in the regions of Van-Agn-Kars-Igdir and to introduce more modern border surveillance systems instead. The Phase-I Project is composed of 3 components: a Component A Clearance: This component covers the mine clearance activities for the selected regions.
o Component B Certification/Quality Control (QA/QC): This component covers the quality assurance, quality control and post-clearance certification to provide confidence (to the beneficiary and the implementation agency) that clearance and quality requirements defined in the standards have been met and that cleared land is indeed safe for use. o Component C: Post-clearance: This component aims to identify the lessons-learned which are relevant to the planning, preparation and clearance phases of the operation through the conduct of a formal post project review (PPR). The project is expected to generate the following results:
o Demining the border regions of Van-Agrt- Igd1r-Kars is completed. o Post project review is conducted. o Demined areas are certified in compliance with the international standards. The project main impact will be observed through the prevention of illegal migration and all types of cross-border crimes at Turkey's Eastern borders in line with EU's IBM policies and strategies via demining the area and providing effective and humanitarian border surveillance tools for a technologically supported modem border surveillance system. The Ministry of National Defence (Turkish Mine Action Center (TURMAC)). Ministry of Interior (Border Management Department) and Turkish General Staff (Land Forces Command) are the main beneficiaries of the Project. UNDP provides technical assistance for the efficient and effective implementation of the Project through the Grant Contract, signed between CFCU (Central Finance and Contracts Unit) and UNDP and endorsed by the Delegation of the European Union to Turkey (El-JD). This Terns or Reference is for the International Consultant for the Post-Project Review of the Socioeconomic Development through Demining and Increasing the Border Surveillance Capacity at the Eastern Borders of Turkey Phase-I ('Project', hereinafter), who will provide technical assistance to the UNDP and TURMAC on preparation of post-project review under the overall guidance and supervision of the UNDP Project Manager.
2 Evaluation Objectives and Scope The objective of the assignment is for the Consultant to provide technical assistance to the UNDP in identifying the lessons-learned which are relevant to the planning, preparation and clearance phases of the project through the conduct of a formal post project review (PPR).
Page | 30
In line with the evaluation criteria of relevance, efficiency, effectiveness and sustainability, the below areas will form the parameters of the evaluation and the areas in which recommendations should be made for a subsequent approach. Strategic Positioning. Concept and Design The evaluation will assess the concept and design of the project, including an assessment of the appropriateness of the objectives, planned outputs, activities and inputs as compared to cost-effective alternatives. Implementation The evaluation will assess the implementation of the intervention in terms of quality and timeliness of inputs and efficiency and effectiveness of activities carried out. Also, the effectiveness of management as well as the quality and timeliness of monitoring and backstopping by all parties to the project should be evaluated, particularly the evaluation is to assess the use of adaptive management. Partnership and Coordination The evaluation will assess effectiveness and appropriateness of the collaborations and partnerships that were established to deliver support to the project. This includes an assessment of the partnerships with key stakeholders, ministries, donors as well as with international partners. The evaluation should draw conclusions about the extent to which the UNDP were effective in coordination. Monitoring. Evaluation and Risk Management A further focus of the evaluation will be on the extent to which adequate monitoring was undertaken throughout the period, and the extent to which evaluation systems were adequate to capture significant developments and inform responsive management. The evaluation will assess how Lessons Learned have been captured and operationalized throughout the project implementation. Rights-based Approach and Gender Mainstreaming The evaluation will assess the extent to which the project sought to strengthen a rights-based approach and the mainstreaming of gender into development efforts. This should look at what measures were taken to this end and how successful those have been in addressing specific rights- and gender-related aspects of the mine action.
3. Evaluation Questions The evaluation should address the following questions among others (to be finalized in scoping phase):
Relevance To what extent was the support to the project by the UNDP based on clearly identifiable development needs as outlined in the government's strategies, international obligations and others? During the evaluation period, what economic, social or political changes have taken place that affected the project? How do these relate to the relevance of the mine action in Turkey? What opportunities are there to better align the support to the changed context and the needs of the beneficiaries?
Page | 31
Effectiveness To what extent were the outputs and outcomes of the project, and the indicators used, successful in guiding the support to have maximum positive impact of the project? How might this be improved in future? What factors have contributed to achieving or not achieving intended outcomes? To what extent have UNDP outputs and assistance contributed to outcomes? To what extent arc the intended beneficiaries satisfied with the results? How well have gender considerations been taken into account?
Efficiency • How cost-effective and time-efficient was the implementation by the UNDP of project activities and outputs in the evaluation period? What measures were taken to ensure competitiveness? • To what extent are the planned funding and timeframe sufficient to achieve the intended outcomes? • How appropriate was the approach taken to organizing clearance activities in terms of competitiveness? How could this be improved?
Partnership and Coordination • How appropriate and effective has the UNDP partnership strategy been? What factors contributed to this effectiveness or ineffectiveness?
Sustainability To what extent will the benefits and outcomes continue after external donor funding ends? What can be done to maximise the likelihood of sustainable outcomes? To what extent has the Government of Turkey increased its ownership of the mine action during the period in question? What impact has this had on external support?
Monitoring & Evaluation and Risk Management To what extent did the results framework allow for relevant monitoring of progress and impact of interventions? How could this be improved, with particular reference to the findings regarding relevance? How accurate was the risk assessment undertaken? How effectively were the risks managed? How effective were the provisions for oversight of the activities?
4. Evaluation Methodology The ultimate design of the PPR methodology will result from consultations between the IC, UNDP and key stakeholders. The project progress and achievements will be tested against following evaluation criteria: Relevance -- the extent to which the activity is suited to local and national development priorities and organizational policies, including changes over time.
Page | 32
Effectiveness the extent to which an objective has been achieved or how likely it is to be achieved. Efficiency the extent to which results have been delivered with the least costly resources possible. Results/impacts the positive and negative, and foreseen and unforeseen, changes to and effects produced by the project, including direct project outputs, short-to medium term outcomes, and longer-term impacts, benefits, replication effects and other, local effects. Sustainability the likely ability of the project to continue to deliver benefits for an extended period of time after completion. Projects need to be environmentally as well as financially and socially sustainable. The Project will be rated against individual criterion of relevance, effectiveness, efficiency and impact/results based on the following scale:
Highly Satisfactory (HS): The project has no shortcomings in the achievement of its objectives. Satisfactory (S): The project has minor shortcomings in the achievement of its objectives. Moderately Satisfactory (MS): The project has moderate shortcomings in the achievement of its objectives. Moderately Unsatisfactory (MU): The project has significant shortcomings in the achievement of its objectives. Unsatisfactory (U) The project has major shortcomings in the achievement of its objectives. Highly Unsatisfactory (HU): The project has severe shortcomings in the achievement of its objectives.
As for sustainability criterion, the evaluator should at the minimum evaluate the "likelihood of sustainability of outcomes at project termination. and provide a rating for this. The following four dimensions or aspects of sustainability should be addressed:
Financial resources: a. Are there any financial risks that may jeopardize sustenance of project outcomes? b. What is the likelihood of financial and economic resources not being available once the donor assistance ends (resources can be from multiple sources, such as the public and private sectors, income generating activities, and trends that may indicate that it is likely that in future there will be adequate financial resources for sustaining project's outcomes)?
Socio-political: a. Are there any social or political risks that may jeopardize sustainability of project outcomes? b. What is the risk that the level of stakeholder ownership will be insufficient to allow for the project outcomes/benefits to be sustained? c. Do the various key stakeholders sec that it is in their interest that the project benefits continue to flow? d. Is there sufficient public stakeholder awareness in support of the long-term objectives of the project?
Page | 33
Institutional framework and governance: a. Do the legal frameworks, policies and governance structures and processes arc in place for sustenance of project benefits? b. While assessing this parameter, also consider if the required systems for accountability and transparency, and the required technical know-how arc in place.
On each of the dimensions of sustainability of the project outcomes will be rated as follows:
• Likely (L): There are no or negligible risks that affect this dimension of sustainability.
• Moderately Likely (ML): There are moderate risks that affect this dimension of sustainability. • Moderately Unlikely (MU): There are significant risks that affect this dimension or sustainability
• Unlikely (U): There are severe risks that affect this dimension of sustainability. Ali the risk dimensions of sustainability are critical. Therefore, overall rating for sustainability will not be higher than the rating of the dimension with lowest ratings. For example, if the project has an 'Unlikely' rating in either of the dimensions then its overall rating cannot be higher than 'Unlikely'.
The IC should develop detailed methodology and work plan for PPR. The evaluation tools and techniques may include, but not limited to:
Data collection and Desk review:
• Interviews with representatives of beneficiaries, donors, contractors, UNDP project management and technical team, and any other stakeholders as deemed necessary.
• Review of data including but not limited to reports, work plans, financial reports, meeting minutes, relevant terms of references, communications materials, audit recommendations, consultant evaluations, and records of communications between stakeholders.
Field Visits:
• Visits to provinces where clearance activities conducted. • Interviews and discussions with field-level clearance operation staff.
Data Analysis:
• Application of triangulation of both qualitative and quantitative methods of analysis to the results of the above data-gathering exercises for data consistency and accuracy. Sample Evaluation Matrix would be:
Page | 34
• Presentation of initial analysis to stakeholders as a means of refinement and quality review through interim report and a workshop.
5. Duties and Responsibilities The PPR will be carried out by the IC. He/She will receive the support of UNDP Country Office and project management team, and will be assisted by a facilitator assigned by UNDP (when needed). The following duties and responsibilities defined for the IC are indicative and subject to further detailing through specific service requests to be made by I-NDP during the course of the contract duration.
Desk review of documents, development of draft methodology and PPR outline; Debriefing with UNDP and key stakeholders, agreement on the methodology, scope and outline of the PPR report; Interviews with all stakeholders; Debriefing UNDP and key stakeholders;
Development and submission of the first Draft PPR Report. The draft will be shared with the key project stakeholders for review and comment, and will serve as background document for the PPR Workshop; Conducting the PPR Workshop;
Finalization and submission of the Final PPR Report through incorporating suggestions received on the draft report.
6. Deliverables The Assignment will include interim and final deliverables, which will be further defined in the specific service requests to be made by UNDP to the IC. The products expected from the IC are as follows:
• Detailed methodology, work plan and outline of PPR; • Interim report with findings; • Lessons learned and recommendations for improvement, including recommendations for the revision of project strategy, approach, outputs and activities, if necessary;
• Recommendations for a strategy for future replication of the project approach for other demining projects;
• Description of best practices, and an "action list" in a certain area of particular importance for the project.
Page | 35
The IC will be responsible to submit the following deliverables.
Estimated Date Estimated Number MilestoneDeliverables of Days to be invested* 13-15 November 2017 3 Desk review, development of methodology, drafting mission programme. Incorporating comments received from UNDP Country Office (if necessary).
16-18 November 2017 3 In-country field visits, interviews, preliminary mission findings, briefing(s), debriefings with project partners.
20-21 November 2017 2 Interviews with beneficiaries. government representatives and donors.
22-27 November 2017 5 Submission of Draft PPR Report
J
27-29 November 2017 2 Preparation for PPR Workshop
30 November 2017 PPR Workshop
1-4 December 2017 3 Finalization of the PPR Report in line with the comments received from the relevant stake holders regarding the Draft PPR Report discussed at the PPR Workshop.
Total Number of days 18
* The number of days may change among different activities and deliverables but the total days to be invested is expected to be maximum 25 days for the content of this TOR. UNDP has the right to request from the Consultant additional number of days to be invested for additional activities, based on the needs of the project. 7. Timing, Duration and Place of Work The IC is expected to invest a maximum 25 days in total to provide the deliverables to be specified in service requests. Estimated contract start date: 13 November 2017 Estimated contract completion date: 4 December 2017 Place of work for the assignment is Ankara and Igdir The Consultant will travel to Igdir to evaluate demining activities.
8. Services and Facilities to be provided by UNDP The principal responsibility for managing this evaluation lies with UNDP Country Office in Turkey. UNDP will assign a facilitator to set up the stakeholder interviews, arrange the field visits, coordinate the background work regarding the PPR Workshop and provide translation (when necessary).
Page | 36
ANNEX TWO
Socio Economic Development through Demining and Increasing the Border Surveillance Capacity at the Eastern Borders of Turkey (Phase One)
PPR Methodology and Programme
1. METHODOLOGY
Purpose
The purpose of the evaluation is to assess the outcomes of the support provided by UNDP to the Government of Turkey to clear border minefields and enhance border surveillance. The evaluation will also extract lessons learned for future mine action activities in Turkey and other UNDP supported mine action programmes.
Background
As per the UN Development Cooperation Strategy the overall objective of the Project has been to contribute to the social and economic development through demining and more secure borders in Eastern Turkey. The specific objective of the Project has been to contribute to the prevention of illegal migration and cross-border crime on Turkey’s eastern borders by clearing the border regions of mines and providing effective humanitarian border surveillance tools for a technologically supported modern border surveillance system. The Executive Agency for the project has been the Ministry of Interior.
The project has aimed to achieve the following outcomes:
Outcome 1: Enabled environment for inclusive and democratic governance, rule of law and access to justice for all, including further compliance with international commitments and human rights norms and standards
Outcome 1.1: Capacity-building programs developed for accountable and responsive governance mechanisms and structures in public institutions, at the national, regional and local levels, grounded on international principles, including internally displaced persons, and in alignment with European Union accession priorities
The project has aimed to achieve the following outputs:
1. Completing the demining of the border regions of Van-Ağrı-Iğdır-Kars 2. Demined areas are certified in compliance with the international and Turkish mine action standards 3. Post project review is conducted.
Desk Review
The evaluation will begin with a desk review of the project document, key policy documents, reviews, work plans, project reports, financials. This analysis will enable the consultant to select a qualitative investigation sample.
The desk review and analysis will also highlight the priority issues to be covered in the qualitative research, the knowledge gaps that need to be addressed in the interviews and refine the categories for the interviews.
Interviews
The interviews will use an appreciative inquiry model which would ask informants to recall times when the program worked particularly well, then identify the approach it exemplified during those times. In addition to the key informants interviews the evaluation will include some semi structured interviews with small groups.
Page | 37
The evaluation will utilise a most significant change approach in these interviews which will be a structured process for generating and selecting stories of change that identify what different individuals and groups see as the most important outcomes from the program.
There needs to be a diversity of perspectives from government, donors, UNDP, project partners, beneficiaries and some gender diversity.
Workshop
A report outline will be identified based on the desk review and initial interviews. An interim report will be prepared and shared with key partners ahead of a workshop to discuss the key questions raised during the evaluation and feed into the report. The results from the workshop will feed into the preparation of the final report.
The Report
The report will document the progress of the project as well as extracting lessons learned. It is therefore planned that this report will be shared with Government, Donors, and the mine action sector to disseminate the findings. The report will inform the development of the next phase of the project and will be shared through a PDF copy of the report.
2. EVALUATION METHODS
Evaluation Question Indicator/ Method Data Performance Source Measure Relevance To what extent was UNDP support based on clearly identifiable Description of Interviews Project development needs as outlined by government strategies, alignment of document document international obligations etc? How appropriate was that design for govt and UNDP review and project UNDP support? strategies reports During the evaluation period what economic social or political Documentation Interviews Project changes have taken place that affected the project? How were of changes and Reports document these relevant to mine action and development in Turkey? What through and project changes were made to the project in response to these changes? interviews and reports reports What opportunities are there to better align the support to the Interview Interviews Interviews changed context and the needs of the beneficiaries? description of and Govt opportunities Strategies Effectiveness To what extent did the project design guide the support and Interview Interviews Project maximise impact? description of and Document , process and document contracts actions review and MOUs, Project reports What factors have contributed to UNDP achieving or not achieving Description of Interviews Project intended outcomes? To what extent has UNDP support factors that have and reports reports contributed to outcomes? Have UNDP tried to do too much? Not led to achieving enough? not achieving outcomes How satisfied are the beneficiaries (Min of National Defence, Discussion with Interviews Interviews Ministry of Interior, Turkish General Staff) with the results? What the beneficiaries and project would they do differently? What does this mean for Phase 2? of the results reports
How well has UNDP’s work been integrated with the activities and Interview Interviews Policy priorities of Government at national and province levels? Can you description of and desk documents, give some examples? What’s worked well? What hasn’t worked process and review Project well? activities document,
Page | 38
project reports How were gender considerations taken into account? How has a Interview Interviews Project human rights based approach featured in this project? describes and desk document process to review and project include gender reports and HRBA How effective has the UNDP partnership strategy been? How Interviews with Interviews Project effective has UNDP been in leading that strategy? What factors UNDP and and desk Document, contributed to it working or not working? partners to review partnership determine strategy effectiveness and project reports What does success look like? What will be required to achieve Discussion with Interviews Workshop that success? all participants and design and workshop workshop Efficiency How cost effective and time efficient was UNDP’s implementation Discussions with Interviews Project of demining and other project activities and outputs? What all partners on and desk document, measures were taken to ensure competitiveness? How successful project activities review project were they? and outputs reports, contracts, MoUs with Govt To what extent were the planned funding and timeframe sufficient Interviews with Interviews Project to achieve the intended outcomes? Did the project try to do too all partners and documents much? Not enough? recommendatio and project ns for workshop reports How appropriate was the approach taken to planning clearance Interviews Interviews Project activities? How well was the competitive process managed? What informed by and document, lessons can be learned and how can this be improved? desk reviews workshop contracts, that explain the design project approach to reports clearance contracts How well were mine clearance activities prioritised? How well did Interviews with Interviews Project survey activities inform the clearance plans? What lessons have partners and and desk document, been learned for future prioritisation? providers that review MoUs, explain contracts prioritisation Impact/Results When was the programme most successful? What contributed to Interviews with Interviews Project that success? all stakeholders and reports that discuss workshop programme design success What have been the most significant changes that have occurred Interviews with Interviews Project for the beneficiaries as a result of the project? How did it happen? partners and and reports providers to workshop assess changes design How has the project improved border surveillance in the eastern Interviews that Interviews Project border areas? What lessons have been learned? discuss how desk review document cleared land has and and reports been handed workshop over and used design for border surveillance How has the project contributed to achieving the EU targets for Interview with Interviews Policy Turkey’s accession into the EU and Turkey’s obligations to the EU and and documents APMBT? Government
Page | 39
about accession workshop process and design APMBT What could UNDP do more of? Less of? What are a few big Interviews with Interviews Workshop changes that could be made to enhance the project? all stakeholders and design workshop design Sustainability What evidence is there that benefits and outcomes will continue Interviews with Interviews Project after donor support ends? all stakeholders and desk document discussing review and project process, reports actions, strategies What evidence is there of increased ownership from GoT? What Interviews Interviews Project impact has this had on external support? discussing document strategies and and project actions reports What needs to be changed to maximise the likelihood of Interviews Interviews, Project sustainable outcomes? What risks are there? discussing desk review document, current and and project possible future workshop reports and strategies and design workshop actions design What is the plan for future resourcing of the project from the GoT Interviews Interviews Correspond and donors? discussing and ence, resourcing workshop meeting strategies with design records and government and workshop donors design Are the legal frameworks, policies and governance structures in Interviews Interviews Project place for sustained impact? How much more needs to be done? discussing legal and desk document, What resources are required (including time)? frameworks, review project policies and reports governance structures plans and actions M&E and Risk Management How well did the results framework enable monitoring progress Interviews with Interviews Project and impact? How could this be improved? stakeholders and desk document, discussing the review results results framework, framework project reports How accurate was the risk assessment? How effectively were the Interviews to Interviews Project risks managed? What are the main risks emerging in the future? discuss risk and desk document, management review of risk risk process assessment assessmen throughout the t, project project and reports future risks How effective were the oversight provisions for clearance Interviews Interviews Project activities? How well has the QA/QC and post clearance discussing and desk document, certification linked in with the demining work? clearance and reviews contracts QA/QC process and project reports
Page | 40
3.PROGRAMME
DATE ACTION 24-26 November 2017 Develop methodology 27 November Desk Review 28 November AM UNDP Staff PM Fly to Igdir 28 November – 30 November Field visits, interviews with government representatives, project 2017 partners, beneficiaries in Igdir, 1 December Meeting with TURMAC and government representatives in Ankara 2 – 3 December 2017 Write draft PPR 4-5 December Interviews with government representatives 6-7 December Write Draft PPR 9 December 2017 PPR Workshop 10-12 December 2017 Finalise PPR and Submit to UNDP
Page | 41
ANNEX THREE
LIST OF INTERVIEWEES
Seher Alacaci - Assistant Resident Representative, UNDP
Alistair Craib - EOD and Mine Action Specialist
Michael Berrill – Chief Advisor Operations, UNDP
Scott Derry – Manager, RPS Turkey
Buche Dundar – Programme Manager, European Union
Col Murat Can – Head of Strategy, TURMAC
Maj Can Ceylan – Acting Head, Quality Management, TURMAC
Emrah Erhun - Finance Associate, UNDP Turkey
Izzet Gerren - Team Leader, CFCU
Sedat Haas – Field Coordinator, UNDP Turkey
Olaf Juergenson – Regional Mine Action Advisor, UNDP
Ben Lark – Former Chief Technical Advisor, UNDP Turkey
Faiz Paktian – Capacity Building Advisor, UNDP Turkey
Ozlem Ozgul Yılmaz – Project Associate, UNDP
Jaap du Preez – Manager, Mechem Turkey
Allan Poston – Project Manager, UNDP Turkey
Serhat Senel – Procurement Associate, UNDP Turkey
Senior Officers – Turkish Land Forces Command
Marty Steel – QA/QC Inspector, RPS Turkey
Johan Strydom – Operations Manager, Mechem Turkey
Secil Turkmenoglu – Team Leader, CFCU
Col Mamet Zaki – Head of Operations, TURMAC
Page | 42
ANNEX FOUR
Glossary
CHA - Confirmed Hazardous Area
EBMCP - Eastern Border Mine Clearance Project
EBMCS - Eastern Border Mine Clearance Standards
EOD - Explosive Ordnance Disposal
HMA - Humanitarian Mine Action
IMAS - International Mine Action Standards
MDD - Mine Detection Dogs
MoD - Ministry of Defence
MoI - Ministry of the Interior
NTS - Non-Technical Survey
ODP - Operational Demining Process
OWG - Operational Working Group
QA - Quality Assurance
QC - Quality Control
SHA - Suspected Hazardous Area
TAT - UNDP Technical Advisory Team
TS - Technical Survey
TURMAC - Turkish Mine Action Centre
Page | 43