Subtitle “Promoting the usage of administrative data in Statistics Estonia by describing and harmonising metadata” Final grant report Table of Contents Executive Summary .............................................................................................................................. 3 List of acronyms .................................................................................................................................... 5 Introduction ........................................................................................................................................... 6 1. Obtaining knowledge about best practices of administrative data and metadata management system from another Member State (study visit) ............................................................................... 7 1.1. Summary and difficulties encountered ................................................................................... 11 2. Analysing and compiling data about current agreements, data sources and data structure descriptions .......................................................................................................................................... 12 2.1. Summary and difficulties encountered ................................................................................... 15 3. Analysing the questionnaires and finding variables that could be replaced by administrative data ....................................................................................................................................................... 15 3.1. Summary and encountered difficulties ................................................................................... 26 4. Mapping management processes of administrative data and metadata in Statistics Estonia .. 28 4.1. Summary and difficulties encountered ................................................................................... 36 5. Creating vision document on how to give feedback to the data owners about data transmission deadlines and agreed data structures ................................................................................................ 36 5.1. Summary and encountered difficulties ................................................................................... 38 6. Describing metadata for the data sources whose cooperation agreements are renewed in the metadata system .................................................................................................................................. 39 6.1. Summary and difficulties encountered ................................................................................... 41 7. Renewing cooperation agreements made with data owners before the year 2010 .................... 42 7.1. Summary and difficulties encountered ................................................................................... 45 References ............................................................................................................................................ 45 2 Executive Summary According to Statistics Estonia’s strategy, our goal is to produce high quality statistics with as low administrative burden and as high efficiency as possible. In order to achieve this, we need to improve the use of administrative data and describe the related metadata in our metadata management system. At the moment, Statistics Estonia uses over 100 different administrative data sources (state registries) in the statistical production process. Managing, describing and improving the related information and metadata of those sources is a challenging and ongoing process. In this project we have described and standardised metadata for the data sources whose cooperation agreement needed updating. During the process we also had the chance to develop and strengthen the partnership with the data owners, which is the key element of using the data of administrative sources. Our project started with learning form Statistics Austria’s experience and we were able to analyse and work through all our administrative data management related information to start managing it more efficiently. Efficient data management is only possible if we have optimized management processes. During the project we were able to map the as-is and to-be processes of administrative data and metadata management. The volume of administrative data and metadata is growing fast, so it is now clear, that we need to move towards more automated processes. For that reason we have created the vision document for developing the new information system Administrative Data Gate, that will allow to send automated feedback and reminders to the data owners and also automate the data checking processes. The grant project enabled us to analyse our current questionnaires domain by domain and make suggestions to use additional administrative data sources to lower the response burden. This analyse was a new approach for us, because usually the statisticians are responsible for their statistical activities. But now we analysed different questionnaires together centrally and had the opportunity to give the statisticians some new ideas, which sources to use and improve the usage of administrative data in our organisation. 3 Statistics Estonia is very grateful for being able to realise the activities in this grant project. It was really helpful that we could have our temporary employee who worked through a lot of information and we were able to start moving towards more automated processes of managing administrative data. 4 List of acronyms ATAO – Statistics Design Department BORA – The Beneficial Owners Register Act EAS – Enterprise Estonia ECAA – Estonian Civil Aviation Authority EMDE – Electronic Maritime Information System GSBPM – Generic Statistical Business Process Model MUIS – System of (Estonian) Museums RIHA – Administration system for the state information system SE – Statistics Estonia sDWH – Statistical Datawarehouse TÖR – Working register 5 Introduction The main objective of this grant project is to improve the use of administrative data sources in Statistics Estonia. According to Statistics Estonia’s strategy our goal is to produce high quality statistics with as low administrative burden and as high efficiency as possible. Producing high quality statistics is possible only when we have standardised metadata and efficient production processes. Harmonised metadata is usable across all statistical domains, which means if one data source is used in different statistical activities the metadata will be described only once. The described metadata will be available directly for the users and for the systems in the live production environment. Statistics Estonia has set the goal to reduce administrative and response burden for the respondents. This is possible only if we use more administrative sources and quit using some of the questionnaires or prefilling some values on the questionnaire to help the respondent to answer. Improving the use of administrative data for the statistical production has one key element – close cooperation and partnership with the data owners. At the moment Statistics Estonia uses over 100 different administrative sources and our goal is to build closer cooperation with the data owners in order to ensure efficient negotiations and high quality data delivery. One important part of the cooperation are valid and up to date data delivery contracts – it is important that both sides of the contract know their responsibilities and that data owners know why the data is needed and that it is securely processed and stored in Statistics Estonia. We have planned several activities during the project that will help to re-use information, produce statistics more efficiently, reduce administrative and response burden. To perform the tasks of the grant project we have conducted weekly project team meetings, where we discuss and agree on the tasks of upcoming week. We are using the web application (JIRA) for assigning and monitoring planned activities. All the team member need to report weekly on the progress of their tasks and possible difficulties, to ensure the execution of the project on schedule. 6 Now the overview of the progress on the tasks of the grant project is given. The overview is written task by task and also the level of progress is evaluated. For every task the encountered difficulties and overall summary are described briefly. 1. Obtaining knowledge about best practices of administrative data and metadata management system from another Member State (study visit) In order to use best practices available in other European statistical offices, we planned to have a study visit at the beginning of the project. In order to choose our possible destinations, we gathered some information from our colleagues, who have attended different Eurostat working groups. We received information that Austria and Finland both have advanced systems for managing metadata and administrative data. Both countries also conduct register based population and household census. Statistics Austria was able to welcome us in October to share their knowledge and experience. And as Austria is considered one of the leaders in European statistical system concerning the use of administrative data, we were happy to plan the 1,5 day agenda for the study visit. The study visit took place from 17 to 19 of October and we had a very full agenda for the 18 and 19 of October. From Statistics Estonia four people attended the visit: two Leading Methodologists from Statistics Design