Policy Group on Statistical Cooperation

EUROPEAN COMMISSION EUROSTAT

Directorate D: External cooperation, communication and key indicators Unit D-1: Statistical Cooperation with European and Mediterranean countries

Doc. PGSC/2010/12

Meeting of the Policy Group on Statistical Cooperation 6 - 7 October 2010

Sarajevo, Hotel Europe Start: 9.30 am

Defining criteria for compliance levels (Turkey)

Point 7 of the Agenda The purpose of this document is to present the view of TurkStat on setting objective criteria for compliance levels in SMIS+.

Background Eurostat Unit D1 - Statistical Cooperation with European and Mediterranean Countries monitors the compliance of candidate and potential candidate countries and some of European Neighbourhood Policy (ENP) countries with the acquis in the field of statistics. Statistical Management Information System (SMIS+) is the web-based tool used in this process. The compliance monitoring is undertaken in cooperation with the National Statistical Institutes (NSIs) of the above mentioned countries once a year. The NSIs are asked to make self-assessments on the developments of the statistical system in their countries by submitting on-line information into SMIS+ with its predefined structure of statistical themes, modules and datasets. Turkey was included in the compliance monitoring exercise for the first time in 2002 with Data Availability (DA) exercise. In 2004, the Compliance Database (CDB) study as a legal compliance monitoring tool was combined with DA, which is a material compliance monitoring tool, under the new instrument SMIS and then SMIS+. The exercise has been implemented on a regular once a year basis since 2004 in Turkish Statistical Institute (TurkStat).

2010 Compliance Monitoring Round The 2010 compliance monitoring round was initiated for all candidate and potential candidate countries at the beginning of February 2010. The countries were asked to provide self- assessments in SMIS+ by the end of February. After the submissions of the self-assessments by countries, Eurostat evaluators from the technical units started to provide their evaluation regarding the self-assessments of the countries. For Turkey, long and short versions of assessment reports including Eurostat comments for 2009 round were received on December 1, 2009. On February 4, 2010, an e-mail from Eurostat was received announcing that Eurostat has launched 2010 Compliance Monitoring Round. Subsequently, the process in TurkStat was initiated. Official letters were sent to the technical departments and e-mails were transmitted to the previous year users (country assessment providers, CAPs) on February 8, 2010. After the assessments of CAPs from TurkStat, Central Bank of the Republic of Turkey, Undersecretariat of Treasury, Ministry of National Education and Ministry of Finance, SMIS+ was confirmed by the Country Leader February 26, 2010 just as envisaged by Eurostat.

Purpose The purpose of setting objective criteria for the compliance levels in SMIS+ are to avoid the major differences between compliance levels made by countries responsible for SMIS+ and compliance evaluations made by Eurostat and to discuss whether objective criteria could be determined for the compliance levels in SMIS+.

Compliance Levels in SMIS+ Compliance levels in SMIS+ are “fully compliant”, “highly compliant”, “medium compliant”, “low compliant” and “not at all compliant”.

2 TurkStat suggests the following definitions for the above mentioned compliance levels:  “ Fully compliant” means the requirements are fully observed with respect to all essential criteria.  “ Highly compliant” means that there are only minor shortcomings, with a large majority of the requirements are being fully met.  “Medium compliant” means that some substantive action has been taken and complies with some of the requirements.  “Low compliant” means that there are major shortcomings, with a large majority of the requirements not being met.  “Not at all compliant” means that requirements are definitely noncompliant with the criteria.  “Not set” means that modules/themes/datasets are not evaluated.

Suggested Criteria for a Checklist Constitution of a checklist might be suggested in order to determine objective criteria for the compliance levels of SMIS+. Firstly, some criteria could be listed, as “coverage of variables and sectors”, “transmission of complete datasets to Eurostat through eDAMIS”, “transmission of qualitative information (metadata, etc.) together with datasets”, “periodicity, timeliness and punctuality”, “classifications, definitions and methods”, etc. All these criteria might have a percentage to be weighted in accordance with its significance. For instance, “coverage of variables and sectors” could be % 30; “periodicity, timeliness and punctuality” could be % 20 and % 10 for the “classifications, definitions and methods”, etc. Besides, intervals for the compliance levels could be determined as % 90 - % 100 for “fully compliant”, % 70 - % 90 for “highly compliant”, etc. And the sum of the percentages could show the compliance level. If the sum of the percentages in the checklist is % 80, the theme / module could be assessed as “highly compliant”.

Conclusion The primary objective of the SMIS+ has been the development of a systematic approach to the assessment and monitoring of the progress complying with the acquis in statistics by candidate, potential candidate and some ENP countries. SMIS+ takes into account the needs of the Commission, participant countries and CAPs and ensures proper follow-up of the compliance during the negotiation process and beyond. It also helps to identify gaps and problems of the national statistical systems. One of the basic problems regarding SMIS+ is the major differences between the compliance levels made by countries and compliance evaluations made by Eurostat. In order to avoid this problem, some objective criteria could be determined and the users could make their assessments in a more objective manner.