Open Data Goldbook for Data Managers and Data Holders Open Data Goldbook for Data Managers and Data Holders Practical guidebook for organisations wanting to publish Open Data Last update: January 2018 www http://www.europeandataportal.eu @ [email protected] Licence: CC-BY 2 Open Data Goldbook for Data Managers and Data Holders Capgemini Consulting prepared this Goldbook as part of the European Data Portal project. The European Data Portal is developed by the European Commission with the support of a consortium led by Capgemini Consulting, including INTRASOFT International, Fraunhofer Fokus, con.terra, Sogeti, the Open Data Institute, Time.Lex and the University of Southampton. For more information about this Goldbook, please contact: European Commission Directorate General for Communications Networks, Content and Technology Unit G.1: Data Policy & Innovation Daniele Rizzi – Policy Officer Email: [email protected] Project team Dinand Tinholt – Vice President, EU Lead, Capgemini Consulting Executive lead European Data Portal Email: [email protected] Wendy Carrara – Principal Consultant, Capgemini Consulting Project Manager European Data Portal Email: [email protected] Written and reviewed by Wendy Carrara, Sem Enzerink, Fréderique Oudkerk, Cosmina Radu, Eva van Steenbergen (Capgemini Consulting) DISCLAIMER By the European Commission, Directorate-General of Communications Networks, Content & Technology. The information and views set out in this publication are those of the author(s) and do not necessarily reflect the official opinion of the Commission. The Commission does not guarantee the accuracy of the data included in this Goldbook. Neither the Commission nor any person acting on the Commission’s behalf may be held responsible for the use, which may be made of the information contained therein. © European Union, 2018. All rights reserved. Certain parts are licensed under conditions to the EU. Reproduction is authorised provided the source is acknowledged. 3 Open Data Goldbook for Data Managers and Data Holders Table of Contents Reading guide .......................................................................................................................................... 6 Glossary ................................................................................................................................................. 10 1. Open Data in a Nutshell ................................................................................................................ 12 1.1. General definition of Open Data ........................................................................................... 12 1.2. Public Sector Information and Open Government Data ....................................................... 13 1.2.1. Is PSI Different from Open (Government) Data? .......................................................... 14 1.3. The benefits of Open Data .................................................................................................... 14 2. How to build an Open Data Strategy ............................................................................................. 19 2.1. Setting the ambition .............................................................................................................. 19 2.1.1. Defining the As-Is Situation ........................................................................................... 19 2.1.2. Define the To-Be situation and define measurable goals ............................................. 21 2.2. Creating your strategy ........................................................................................................... 22 2.2.1. Publishing data: ‘Open by default’ or ‘Closed unless’? ................................................. 23 2.3. Drafting an Open Data Policy ................................................................................................ 23 2.3.1. The Open Data Policy: an overview ............................................................................... 24 2.3.2. The content of your policy ............................................................................................ 24 2.4. Overcoming barriers in publishing Open Data ...................................................................... 27 2.4.1. Ensuring organisational alignment as a Key Success Factor ......................................... 27 2.5. Critical success factors ........................................................................................................... 28 2.5.1. Critical success factors for publishing ........................................................................... 29 2.5.2. Critical success factors for re-use .................................................................................. 29 3. Technical preparation and implementation .................................................................................. 30 3.1. Data management ................................................................................................................. 30 3.1.1. Data management per instance .................................................................................... 30 3.1.2. Federated Data Management with a Centre of Expertise (CoE) ................................... 31 3.1.3. Fully Centralised Master Data Management ................................................................ 32 3.1.4. Implementing Master Data Management ..................................................................... 32 3.2. Extract, transform, and publish ............................................................................................. 32 3.3. Channels ................................................................................................................................ 33 3.3.1. Web download .............................................................................................................. 34 3.3.2. Data portal ..................................................................................................................... 34 3.3.3. API.................................................................................................................................. 34 3.4. Search .................................................................................................................................... 34 4 Open Data Goldbook for Data Managers and Data Holders 3.4.1. Basic search ................................................................................................................... 34 3.4.2. SPARQL .......................................................................................................................... 35 3.5. Pre-requisites, choices and accountability ............................................................................ 35 4. Putting in place an Open Data lifecycle ......................................................................................... 37 4.1. Collecting data ....................................................................................................................... 37 4.1.1. General collection process ............................................................................................ 37 4.1.2. Data quality ................................................................................................................... 41 4.1.3. Preparing the data: technical openness ........................................................................ 42 4.1.3.1. Linked Data ................................................................................................................ 42 4.1.3.2. Metadata ................................................................................................................... 46 4.1.3.3. The 5-Star Open Data model ..................................................................................... 50 4.1.4. Preparation: legal openness .......................................................................................... 52 4.1.5. The Final Check .............................................................................................................. 53 4.2. Publishing data ...................................................................................................................... 54 4.2.1. Publishing as files on a website ..................................................................................... 54 4.2.2. Upload to a portal ......................................................................................................... 54 4.2.3. Publish through an API .................................................................................................. 55 4.3. Maintaining data ................................................................................................................... 56 4.3.1. Maintaining data and metadata regularly ..................................................................... 56 4.3.2. Checking URIs & URLs .................................................................................................... 56 4.3.3. Checking user feedback and continuous improvement ................................................ 56 5. Ensuring and monitoring success .................................................................................................. 57 5.1. Engaging re-users .................................................................................................................. 57 5.2. Monitoring your Open Data initiative ..................................................................................