M GIS

A Short Introduction to Volunteered Geographic Information Presentation of the OpenStreetMap Project

Sylvain Bouveret – LIG-STeamer / Universit´eGrenoble-Alpes

Quatri`eme Ecole´ Th´ematique du GDR Magis. S`ete, September 29 – October 3, 2014 Sources

I Part of the presentation dedicated to OSM inspired from:

I An old joint presentation with N. Petersen and Ph. Genoud

I Nicolas Moyroud: Several talks from 3rd MAGIS summer school 2012 Released under licence CC-BY-SA and downloadable here: http://libreavous.teledetection.fr.

I Guillaume All`egre: Cartographie libre du monde: OpenStreetMap Released under licence CC-BY-SA.

I Reference book about VGI [Sui et al., 2013]

I Other references cited throughout the presentation

Sui, D. Z., Elwood, S., and Goodchild, M., editors (2013). Crowdsourcing geographic knowledge: Volunteered Geographic Information (VGI) in Theory and Practice. Springer.

´ M GIS 2 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Outline

1. Introduction to Volunteered Geographic Information

2. Presentation of the OpenStreetMap Project

3. Using OpenStreetMap Data

4. Using Volunteered Geographic Information

´ M GIS 3 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Outline

1. Introduction to to Volunteered Volunteered Geographic Geographic Information Information

2. Presentation of the OpenStreetMap Project

3. Using OpenStreetMap Data

4. Using Volunteered Geographic Information

´ M GIS 3 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Outline

1. Introduction to Volunteered Geographic Information

2. Presentation of of the the OpenStreetMap OpenStreetMap Project Project

3. Using OpenStreetMap Data

4. Using Volunteered Geographic Information

´ M GIS 3 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Outline

1. Introduction to Volunteered Geographic Information

2. Presentation of the OpenStreetMap Project

3. Using OpenStreetMap OpenStreetMap Data Data

4. Using Volunteered Geographic Information

´ M GIS 3 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Outline

1. Introduction to Volunteered Geographic Information

2. Presentation of the OpenStreetMap Project

3. Using OpenStreetMap Data

4. Using Volunteered Volunteered Geographic Geographic Information Information

´ M GIS 3 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete First part

1 Introduction to Volunteered Geographic Information

Beyond traditional GIS

A new trend

Some examples

Features of participative datasets

Volunteered vs Contributed

Open vs Closed

Sensing vs Thinking

Volunteered Geographic Information Introduction to Volunteered Geographic Information

Beyond traditional GIS A new trend Some examples

Features of participative datasets Volunteered vs Contributed Open vs Closed Sensing vs Thinking

Volunteered Geographic Information

´ M GIS 5 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I 2004: Participative data, geographic crowdsourcing, volunteered geographic information, neogeographic datasets...

A very brief history of GIS

First users of GIS (at least in France): local authorities, department of defense

I Early 90’s: paper maps (unprecise, to be regularly updated...)

I 1990 → 2010: Digital transposition of data

I 2000: Integration to enterprise IS (first spatial extensions to Oracle and Postgres)

I 2002: Geospatial webservers + OGC standards

I 2005: Mobility

´ M GIS 6 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete A very brief history of GIS

First users of GIS (at least in France): local authorities, department of defense

I Early 90’s: paper maps (unprecise, to be regularly updated...)

I 1990 → 2010: Digital transposition of data

I 2000: Integration to enterprise IS (first spatial extensions to Oracle and Postgres)

I 2002: Geospatial webservers + OGC standards

I 2005: Mobility

I 2004: Participative data, geographic crowdsourcing, volunteered geographic information, neogeographic datasets...

´ M GIS 6 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete OpenStreetMap

http://www.openstreetmap.org/

I Project started on 2004

I Open and collaborative geographical database of the world

I Content generated by users (about 1.8M registered users)

I Free license (initially CC-by-sa; ODbL since 2012)

More about OpenStreetMap later

´ M GIS 7 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Wikimapia

http://wikimapia.org/

I Project started on 2006

I Aims at “marking all geographical objects in the world and providing a useful description of them”

I Mostly provides a way for users to give annotations about places in the world, (initially) using Google Maps as a base layer.

I Free license since 2010 (CC-by-SA).

´ M GIS 8 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Google Map Maker

http://www.google.com/mapmaker/

I Project started on 2008

I Equips Google Maps with a map edition interface

I Every registered user can submit modifications

I Modifications have to be approved before being published in Google Maps

I Data released under proprietary license

´ M GIS 9 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Ushahidi

http://www.ushahidi.com/

I Free Software and platform for crisis management

I Crowdsourcing-based mapping

I Focuses on information flow (smartphones, SMS,...)

I Web platform Based on OpenStreetMap and Google Maps for Geocoding (source: Wikipedia).

´ M GIS 10 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Ligue de Protection des Oiseaux

http://www.ornitho.fr/

I A network of“participatory science websites”dedicated to wildlife inventory

I Anyone can participate by adding observations to the database

I Requires some basic knowledge about different species

I In general, no verification is made, except for outliers

I Search engine and visualization tool (map) on the website

´ M GIS 11 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Other examples

I Geolabeled Flickr Images ( http://www.flickr.com/ )

I Foursquare ( http://foursquare.com/ )

I UCrime ( http://ucrime.com/ )

I ...

´ M GIS 12 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Introduction to Volunteered Geographic Information

Beyond traditional GIS A new trend Some examples

Features of participative datasets Volunteered vs Contributed Open vs Closed Sensing vs Thinking

Volunteered Geographic Information

´ M GIS 13 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Some strong common points...

I Data contributed on a voluntary basis by users

I Geospatial nature of data (or at list a part of it)

...But very different features as well:

I Aims

I Geospatial as a first-class citizen or not

I Skills required

I Process for quality assessment (data verification)

I Data license

Features

All these applications are examples of geographical crowdsourcing approaches

´ M GIS 14 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Features

All these applications are examples of geographical crowdsourcing approaches

Some strong common points...

I Data contributed on a voluntary basis by users

I Geospatial nature of data (or at list a part of it)

...But very different features as well:

I Aims

I Geospatial as a first-class citizen or not

I Skills required

I Process for quality assessment (data verification)

I Data license

´ M GIS 14 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete But... What about geolocalized data transmitted by a smartphone, (more or less) unbeknownst to its user?

; An example of crowdsourced geospatial data, assuredly not volunteered!

To volunteer or to contribute?

In all these examples, data is jointly produced by users volunteering to contribute (geographical crowdsourcing)

´ M GIS 15 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete ; An example of crowdsourced geospatial data, assuredly not volunteered!

To volunteer or to contribute?

In all these examples, data is jointly produced by users volunteering to contribute (geographical crowdsourcing)

But... What about geolocalized data transmitted by a smartphone, (more or less) unbeknownst to its user?

´ M GIS 15 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete To volunteer or to contribute?

In all these examples, data is jointly produced by users volunteering to contribute (geographical crowdsourcing)

But... What about geolocalized data transmitted by a smartphone, (more or less) unbeknownst to its user?

; An example of crowdsourced geospatial data, assuredly not volunteered!

´ M GIS 15 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Opt-in vs opt-out

Two approaches to crowdsourced geographic data [Harvey, 2013]:

I Volunteered Geographic Information (opt-in):

I Clarity about purposes

I Control over data collection

I Some guarantees about data reuse

I Contributed Geographic Information (opt-out):

I Unclear purposes

I No (or little) control over data collection

I No control over data reuse

Harvey, F. (2013). To volunteer or to contribute locational information? Towards truth in labelling for crowdsourced geographic information. In Crowdsourcing Geographic Knowledge [...], chapter 3. Springer.

´ M GIS 16 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Two antagonistic examples:

I Google Map Maker: Google owns the data, releases it under proprietary license, whose conditions can change whenever it wants (cf April 2011) irrespective of whether the user is a regular contributor or not

I OpenStreetMap: Data belongs to the contributors, and is released under a free license

Data reuse – licenses

A key distinction between opt-in and opt-out: control over data reuse ...raises the following crucial questions:

I Who owns the data jointly produced by users?

I Under which license this data can be used?

´ M GIS 17 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Data reuse – licenses

A key distinction between opt-in and opt-out: control over data reuse ...raises the following crucial questions:

I Who owns the data jointly produced by users?

I Under which license this data can be used?

Two antagonistic examples:

I Google Map Maker: Google owns the data, releases it under proprietary license, whose conditions can change whenever it wants (cf April 2011) irrespective of whether the user is a regular contributor or not

I OpenStreetMap: Data belongs to the contributors, and is released under a free license

´ M GIS 17 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete But... How to do it while still being compatible with author’s right → freely release data while protecting it and its authors?

I The software community has a solution: use Free licenses

I Initially dedicated to software (like GNU/GPL) → ill-suited for other kinds of intellectual stuff (music, books, pictures, information...)

Geomatics and open (free) licenses

I In geomatics, most datasets are released under proprietary licenses, even those who are funded by public money – IGN, Ordnance Survey,...

I Even if it is free (cost-less), cannot be freely exploited (e.g for scientific projects).

I ; Some VGI projects like OSM explicitly aim at providing free and open geospatial data.

´ M GIS 18 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I The software community has a solution: use Free licenses

I Initially dedicated to software (like GNU/GPL) → ill-suited for other kinds of intellectual stuff (music, books, pictures, information...)

Geomatics and open (free) licenses

I In geomatics, most datasets are released under proprietary licenses, even those who are funded by public money – IGN, Ordnance Survey,...

I Even if it is free (cost-less), cannot be freely exploited (e.g for scientific projects).

I ; Some VGI projects like OSM explicitly aim at providing free and open geospatial data.

But... How to do it while still being compatible with author’s right → freely release data while protecting it and its authors?

´ M GIS 18 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Geomatics and open (free) licenses

I In geomatics, most datasets are released under proprietary licenses, even those who are funded by public money – IGN, Ordnance Survey,...

I Even if it is free (cost-less), cannot be freely exploited (e.g for scientific projects).

I ; Some VGI projects like OSM explicitly aim at providing free and open geospatial data.

But... How to do it while still being compatible with author’s right → freely release data while protecting it and its authors?

I The software community has a solution: use Free licenses

I Initially dedicated to software (like GNU/GPL) → ill-suited for other kinds of intellectual stuff (music, books, pictures, information...)

´ M GIS 18 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete 4 options Attribution (BY) the original author has to be credited Non Commercial (NC) no commercial profit allowed No Derivatives (ND) no derived work allowed Share Alike (SA) derivatives must be licensed under identical terms Six possible combinations CC-by, CC-by-sa, CC-by-nc, CC-by-nc-nd, CC-by-nc-sa, CC-by-nd

Data and open licenses: Creative Commons

First option for geomatics: Creative Commons licenses (artworks)

http://creativecommons.org/

´ M GIS 19 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Data and open licenses: Creative Commons

First option for geomatics: Creative Commons licenses (artworks)

http://creativecommons.org/

4 options Attribution (BY) the original author has to be credited Non Commercial (NC) no commercial profit allowed No Derivatives (ND) no derived work allowed Share Alike (SA) derivatives must be licensed under identical terms Six possible combinations CC-by, CC-by-sa, CC-by-nc, CC-by-nc-nd, CC-by-nc-sa, CC-by-nd

´ M GIS 19 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Data and open licenses: ODbL

I Initial OSM data was released under the terms of the CC-by-sa license.

I However: it turned out that it was not very adapted...

I Combining OSM data with other datasets

I Share-Alike only applies to rendered maps (tiles), not to data itself

I Attribution ; too many contributors!

I Uncertainty about derived work

I After two years of effort, OSM switched to ODbL in September 2012: Attribution, Share Alike, Redistribution (as long as one of the redistributed versions is kept open).

I All the past contributors have been contacted...

I most of them agreed with the new terms

I some of them explicitly disagreed (→ data erased)

I some of them did not answer (→ data erased)

http://opendatacommons.org/licenses/odbl/

´ M GIS 20 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I VGI as geographic citizen science (even if not every VGI application falls into this category)

I The ladder of implication according to Haklay [2013]

I Level 1: Crowdsourcing (citizens as sensors, volunteered computing)

I Level 2: Distributed intelligence (basic interpreters, volunteered thinking)

I Level 3: Participatory science (implication in problem definition and data collection)

I Level 4: Extreme citizen science (problem definition, data collection and analysis)

Sensing vs Thinking

I Different applications ; different skills and levels of implication required from the users

Haklay, M. (2013). Citizen science and volunteered geographic information: Overview and typology of participation. In Crowdsourcing Geographic Knowledge [...], chapter 7. Springer.

´ M GIS 21 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I The ladder of implication according to Haklay [2013]

I Level 1: Crowdsourcing (citizens as sensors, volunteered computing)

I Level 2: Distributed intelligence (basic interpreters, volunteered thinking)

I Level 3: Participatory science (implication in problem definition and data collection)

I Level 4: Extreme citizen science (problem definition, data collection and analysis)

Sensing vs Thinking

I Different applications ; different skills and levels of implication required from the users

I VGI as geographic citizen science (even if not every VGI application falls into this category)

Haklay, M. (2013). Citizen science and volunteered geographic information: Overview and typology of participation. In Crowdsourcing Geographic Knowledge [...], chapter 7. Springer.

´ M GIS 21 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Sensing vs Thinking

I Different applications ; different skills and levels of implication required from the users

I VGI as geographic citizen science (even if not every VGI application falls into this category)

I The ladder of implication according to Haklay [2013]

I Level 1: Crowdsourcing (citizens as sensors, volunteered computing)

I Level 2: Distributed intelligence (basic interpreters, volunteered thinking)

I Level 3: Participatory science (implication in problem definition and data collection)

I Level 4: Extreme citizen science (problem definition, data collection and analysis)

Haklay, M. (2013). Citizen science and volunteered geographic information: Overview and typology of participation. In Crowdsourcing Geographic Knowledge [...], chapter 7. Springer.

´ M GIS 21 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Introduction to Volunteered Geographic Information

Beyond traditional GIS A new trend Some examples

Features of participative datasets Volunteered vs Contributed Open vs Closed Sensing vs Thinking

Volunteered Geographic Information

´ M GIS 22 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Volunteered Geographic Information [Goodchild, 2007] Volunteered geographic information is the harnessing of tools to create, as- semble, and disseminate geographic data provided voluntarily by individuals.

Goodchild, M. F. (2007). Citizens as sensors: the world of volunteered geography. GeoJournal, 69(4):211–221.

What is VGI?

As we have seen, crowdsourcing is just one feature of VGI

´ M GIS 23 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete What is VGI?

As we have seen, crowdsourcing is just one feature of VGI

Volunteered Geographic Information [Goodchild, 2007] Volunteered geographic information is the harnessing of tools to create, as- semble, and disseminate geographic data provided voluntarily by individuals.

Goodchild, M. F. (2007). Citizens as sensors: the world of volunteered geography. GeoJournal, 69(4):211–221.

´ M GIS 23 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Neogeographic datasets

Coote and Rackham [2008] propose the following (complementary) characterization of neogeographic datasets:

I Creation stimulated by lack of available data or restrictions, costs, limitations of conventional data sources

I Involve geographic information provided voluntarily by individuals

I Creation and management are not necessarily ruled by accepted standards

I Data licensed using open-source approach

Coote, A. and Rackham, L. (2008). Neogeographic data quality — is it an issue? In AGI Geocommunity conference, ConsultingWhere Ltd.

´ M GIS 24 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The big picture?

´ M GIS 25 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The big picture?

Participative

´ M GIS 25 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The big picture?

Participative

Geospatial

´ M GIS 25 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The big picture?

Participative

Citizen science

Geospatial

´ M GIS 25 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The big picture?

Participative

Citizen science Open data

Geospatial

´ M GIS 25 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The big picture?

Participative

Citizen science VGI? Open data

Geospatial

´ M GIS 25 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Second part

2 Presentation of the OpenStreetMap project

OSM: History and principles

What is OSM?

History

Technical considerations

Data model

The OSM ontology

Contributing Presentation of the OpenStreetMap project

OSM: History and principles What is OSM? History

Technical considerations Data model The OSM ontology Contributing

´ M GIS 27 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete OpenStreetMap: Genesis

OpenStreetMap was created in July 2004 by Steve Coast, which was then studying at the University College of London.

I He did not understand why the Ordnance Survey created massive geographical datasets but did not freely distribute them to those who had paid to create them (i.e happy tax payers).

I NB: the same thing happens in almost every country in the world (except USA and the Netherlands)

He then decided to start a mapping project whose aim would be to freely provide data to the users.

´ M GIS 28 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete OpenStreetMap: principles

I Collaborative:

I mainly individual contributions

I the more contributors, the more complete the world coverage

I Open: data freely usable without restriction (ODbL).

I Data: OpenStreetMap is not a map, it is a database.

I Online “map” only provided for visualization purposes

I No airborne or satellite view

´ M GIS 29 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete One dataset, several maps

Mapnik standard style

´ M GIS 30 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete One dataset, several maps

Transport map

´ M GIS 30 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete One dataset, several maps

Cycle map

´ M GIS 30 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete One dataset, several maps

MapQuest

´ M GIS 30 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete One dataset, several maps

FranceTopo.fr (enriched with other public datasets such as Nasa SRTM)

´ M GIS 30 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete One dataset, several maps

3D-OSM (XNavigator – University of Bonn and Heidelberg)

´ M GIS 30 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete History

1. Founding and Early History

I 9th August 2004 - openstreetmap.org registered by Steve Coast

I 20th August 2004 - Steve Coast presented his mapping idea at EuroFOO

I 2nd September 2004 - First posting to the mailing list

I 17th July 2005 - Map Limehouse the first Mapping Party

I 22th January 2006 - Release of version 1.0 of the offline editor JOSM

I 20th August 2006 - OpenStreetMap Foundation registered

I 10th November 2006 - Mapnik rendered Slippy map makes its debut.

I 4th December 2006 - Yahoo! aerial imagery sketching allowed

http://wiki.openstreetmap.org/wiki/History

´ M GIS 31 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete History

2. The Start of OSM’s Current Technology Stack

I 5th May 2007 - 1st version of the Potlatch editor.

I 14th-15th July 2007 - First conference, ”State Of The Map 2007”, held in Manchester.

I September 2007 - TIGER data import for the US started

I 20th September 2007 - AND Data for The Netherlands imported

I January 2009 - The French Direction g´en´erale des finances publiques officially allows the OSM contributors to use the Cadastre as a source of data.

http://wiki.openstreetmap.org/wiki/History

´ M GIS 32 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete History

3. The Switch to API 0.6 and the explosion of User Growth

I 21st April 2009 - Big switch to API version 0.6

I 1st April 2010 - Ordnance Survey Opendata releases. OSM partly responsible for bringing this about.

I 30th November 2010 - Use of Bing vertical aerial imagery allowed

I 25th November 2011 - Association OpenStreetMap France registered

I 12th Sept 2012 - License switched over to ODbL

http://wiki.openstreetmap.org/wiki/History

´ M GIS 33 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Number of contributors

2e+06 Today 1.8e+06

1.6e+06

1.4e+06

1.2e+06

1e+06

800000

Number of contributors 600000

400000

200000

0 2000/01 2002/01 2004/01 2006/01 2008/01 2010/01 2012/01 2014/01 2016/01 Date

http://wiki.openstreetmap.org/wiki/Statistics

´ M GIS 34 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Paris

Paris in OpenStreetMap september 2006 −→ october 2010.

http://wiki.openstreetmap.org/wiki/Historical_Coverage

´ M GIS 35 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Current state

Current state of the database (25th September, 2014)

Number of users 1,800,453 Number of uploaded GPS points 4,218,137,961 Number of nodes 2,535,804,643 Number of ways 253,523,371 Number of relations 2,818,286

´ M GIS 36 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Presentation of the OpenStreetMap project

OSM: History and principles What is OSM? History

Technical considerations Data model The OSM ontology Contributing

´ M GIS 37 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete More precisely, each OSM entity has:

I a numeric identifier: OSM ID

I a geometry

I a set of generic attributes present for every element

I uid, user: user id and user name

I timestamp: time of the last modification

I visible: if false then the element should only be returned by history calls

I version: edit version of the object (starts from 1)

I changeset: the changeset (group of edits made within a certain time by one user) in which the object was created or updated

I a set of tags (attributes): key-value pairs

Basic OSM data model

As in most GIS, each geographical entity is described in OSM using:

I Geographical information (geometries)

I Attributes (≈ semantics)

´ M GIS 38 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Basic OSM data model

As in most GIS, each geographical entity is described in OSM using:

I Geographical information (geometries)

I Attributes (≈ semantics)

More precisely, each OSM entity has:

I a numeric identifier: OSM ID

I a geometry

I a set of generic attributes present for every element

I uid, user: user id and user name

I timestamp: time of the last modification

I visible: if false then the element should only be returned by history calls

I version: edit version of the object (starts from 1)

I changeset: the changeset (group of edits made within a certain time by one user) in which the object was created or updated

I a set of tags (attributes): key-value pairs

´ M GIS 38 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Geometries in OpenStreetMap

Three kinds of geometries:

1. nodes : basic element. Geographic point: latitude & longitude (WGS84) → Point Of Interest (POIs)

2. ways : ordered interconnection of nodes open ways → linear features (roads, railways...) closed ways → areas

3. relations : group of any primitive with associated roles Relate nodes, ways and potentially other relations to each other, thereby forming complex objects (e.g. multipolygons). → rela- tionship between objects and abstract objects

´ M GIS 39 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Example of a way

´ M GIS 40 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Example of a relation

´ M GIS 41 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I A tag is a key-value pair

I A key broadly describes an element (e.g. highway, name)

I A value specifically describes its accompanying key

http://wiki.openstreetmap.org/wiki/Tagging_samples/out_of_town

I Use of keys and values is unrestricted (free text)

I the data model is infinitely extensible

I anyone can define and use its own keys and values

Tags in OpenStreetMap

Attributes of geographical entities are described using tags

´ M GIS 42 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I Use of keys and values is unrestricted (free text)

I the data model is infinitely extensible

I anyone can define and use its own keys and values

Tags in OpenStreetMap

Attributes of geographical entities are described using tags

I A tag is a key-value pair

I A key broadly describes an element (e.g. highway, name)

I A value specifically describes its accompanying key

http://wiki.openstreetmap.org/wiki/Tagging_samples/out_of_town

´ M GIS 42 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Tags in OpenStreetMap

Attributes of geographical entities are described using tags

I A tag is a key-value pair

I A key broadly describes an element (e.g. highway, name)

I A value specifically describes its accompanying key

http://wiki.openstreetmap.org/wiki/Tagging_samples/out_of_town

I Use of keys and values is unrestricted (free text)

I the data model is infinitely extensible

I anyone can define and use its own keys and values

´ M GIS 42 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy. Anarchy leads to useless data. Unrestricted tags are the path to the dark side.

Actually, tagging in OSM is governed by an agile self-organizing community process defining the basic ontology of OSM

Resources to find an appropriate tag or explore tag usage:

I Map Features ( http://wiki.openstreetmap.org/wiki/Map_Features ) ; an extensive list of the most commonly used tags

I Taginfo ( http://taginfo.openstreetmap.org/ ) ; a useful site to exploring current tag usage, including tag values that are not necessarily documented

Anarchy?

Unrestricted tags

´ M GIS 43 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Uncontrolled data production process leads to anarchy. Anarchy leads to useless data. Unrestricted tags are the path to the dark side.

Actually, tagging in OSM is governed by an agile self-organizing community process defining the basic ontology of OSM

Resources to find an appropriate tag or explore tag usage:

I Map Features ( http://wiki.openstreetmap.org/wiki/Map_Features ) ; an extensive list of the most commonly used tags

I Taginfo ( http://taginfo.openstreetmap.org/ ) ; a useful site to exploring current tag usage, including tag values that are not necessarily documented

Anarchy?

Unrestricted tags lead to uncontrolled data production process.

´ M GIS 43 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Anarchy leads to useless data. Unrestricted tags are the path to the dark side.

Actually, tagging in OSM is governed by an agile self-organizing community process defining the basic ontology of OSM

Resources to find an appropriate tag or explore tag usage:

I Map Features ( http://wiki.openstreetmap.org/wiki/Map_Features ) ; an extensive list of the most commonly used tags

I Taginfo ( http://taginfo.openstreetmap.org/ ) ; a useful site to exploring current tag usage, including tag values that are not necessarily documented

Anarchy?

Unrestricted tags lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy.

´ M GIS 43 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Unrestricted tags are the path to the dark side.

Actually, tagging in OSM is governed by an agile self-organizing community process defining the basic ontology of OSM

Resources to find an appropriate tag or explore tag usage:

I Map Features ( http://wiki.openstreetmap.org/wiki/Map_Features ) ; an extensive list of the most commonly used tags

I Taginfo ( http://taginfo.openstreetmap.org/ ) ; a useful site to exploring current tag usage, including tag values that are not necessarily documented

Anarchy?

Unrestricted tags lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy. Anarchy leads to useless data.

´ M GIS 43 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Actually, tagging in OSM is governed by an agile self-organizing community process defining the basic ontology of OSM

Resources to find an appropriate tag or explore tag usage:

I Map Features ( http://wiki.openstreetmap.org/wiki/Map_Features ) ; an extensive list of the most commonly used tags

I Taginfo ( http://taginfo.openstreetmap.org/ ) ; a useful site to exploring current tag usage, including tag values that are not necessarily documented

Anarchy?

Unrestricted tags lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy. Anarchy leads to useless data. Unrestricted tags are the path to the dark side.

´ M GIS 43 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Resources to find an appropriate tag or explore tag usage:

I Map Features ( http://wiki.openstreetmap.org/wiki/Map_Features ) ; an extensive list of the most commonly used tags

I Taginfo ( http://taginfo.openstreetmap.org/ ) ; a useful site to exploring current tag usage, including tag values that are not necessarily documented

Anarchy?

Unrestricted tags lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy. Anarchy leads to useless data. Unrestricted tags are the path to the dark side.

Actually, tagging in OSM is governed by an agile self-organizing community process defining the basic ontology of OSM

´ M GIS 43 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Anarchy?

Unrestricted tags lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy. Anarchy leads to useless data. Unrestricted tags are the path to the dark side.

Actually, tagging in OSM is governed by an agile self-organizing community process defining the basic ontology of OSM

Resources to find an appropriate tag or explore tag usage:

I Map Features ( http://wiki.openstreetmap.org/wiki/Map_Features ) ; an extensive list of the most commonly used tags

I Taginfo ( http://taginfo.openstreetmap.org/ ) ; a useful site to exploring current tag usage, including tag values that are not necessarily documented

´ M GIS 43 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Example of OSM tags

I Tags to describe real world objects

I building = church, hotel, school, university...

I highway = motorway, primary, secondary...

I ...

I Tags to describe immaterial objects

I boundary = administrative, national_park...

I ...

I Commons tags

I name = *

I source = *

´ M GIS 44 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete OSM tags

´ M GIS 45 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The price to pay is poor semantics

I harder to detect and fix logical inconsistencies

I less expressive queries

I link to other datasets? (Linked Data)

Some attempts to provide a formal (ontological) backbone to the OSM ontology: e.g. LinkedGeoData

A more formal OSM ontology?

Simple tag structure and unrestricted tags are probably one reason for the success of OSM (easiness of contribution, flexibility, extensibility...)

´ M GIS 46 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete A more formal OSM ontology?

Simple tag structure and unrestricted tags are probably one reason for the success of OSM (easiness of contribution, flexibility, extensibility...)

The price to pay is poor semantics

I harder to detect and fix logical inconsistencies

I less expressive queries

I link to other datasets? (Linked Data)

Some attempts to provide a formal (ontological) backbone to the OSM ontology: e.g. LinkedGeoData

´ M GIS 46 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I Id ( http://ideditor.com/ )

I JavaScript online editor, web application

I easy to use, recommended for beginners

I JOSM ( http://josm.openstreetmap.de/ ) Java OpenStreetMap Editor

I desktop application written in Java, with a plugin architecture

I for advanced users, large set of features and tools

I Potlatch ( http://wiki.openstreetmap.org/wiki/Potlatch_2 )

I written in Flash, can be used directly from a web browser

I has been made obsolete by Id

I Merkaartor (http://merkaartor.be/)

I desktop application C++,Qt (Windows, GNU/Linux, MacOSX)

I Plugins for:

I QGIS ( http://esriosmeditor.codeplex.com/ )

I ArcGIS ( http://esriosmeditor.codeplex.com/ )

OSM Editing tools

How to contribute?

´ M GIS 47 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I Potlatch ( http://wiki.openstreetmap.org/wiki/Potlatch_2 )

I written in Flash, can be used directly from a web browser

I has been made obsolete by Id

I Merkaartor (http://merkaartor.be/)

I desktop application C++,Qt (Windows, GNU/Linux, MacOSX)

I Plugins for:

I QGIS ( http://esriosmeditor.codeplex.com/ )

I ArcGIS ( http://esriosmeditor.codeplex.com/ )

OSM Editing tools

How to contribute?

I Id ( http://ideditor.com/ )

I JavaScript online editor, web application

I easy to use, recommended for beginners

I JOSM ( http://josm.openstreetmap.de/ ) Java OpenStreetMap Editor

I desktop application written in Java, with a plugin architecture

I for advanced users, large set of features and tools

´ M GIS 47 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete OSM Editing tools

How to contribute?

I Id ( http://ideditor.com/ )

I JavaScript online editor, web application

I easy to use, recommended for beginners

I JOSM ( http://josm.openstreetmap.de/ ) Java OpenStreetMap Editor

I desktop application written in Java, with a plugin architecture

I for advanced users, large set of features and tools

I Potlatch ( http://wiki.openstreetmap.org/wiki/Potlatch_2 )

I written in Flash, can be used directly from a web browser

I has been made obsolete by Id

I Merkaartor (http://merkaartor.be/)

I desktop application C++,Qt (Windows, GNU/Linux, MacOSX)

I Plugins for:

I QGIS ( http://esriosmeditor.codeplex.com/ )

I ArcGIS ( http://esriosmeditor.codeplex.com/ )

´ M GIS 47 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The Id editor

´ M GIS 48 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The Id editor

´ M GIS 48 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The Id editor

´ M GIS 48 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The Id editor

´ M GIS 48 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Typical JOSM edition session

´ M GIS 49 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Typical JOSM edition session

´ M GIS 49 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Typical JOSM edition session

´ M GIS 49 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Typical JOSM edition session

´ M GIS 49 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Third part

3 Using OpenStreetMap data

Exploiting the data

Retrieving data (basic principles)

OSM API

Other querying tools

End user applications

Tutorial: create your own database

Basic principles

A step by step example Using OpenStreetMap data

Exploiting the data Retrieving data (basic principles) OSM API Other querying tools End user applications

Tutorial: create your own database Basic principles A step by step example

´ M GIS 51 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete OSM and server applications

I The OSM server cannot be queried for data directly (only a small amount).

I Instead, it can provide some dumps and regular diffs → application servers have to instantiate the DB locally.

I The OSM server can provide tiles updated on a regular basis.

planet.osm (first import)

OSM server http://www. regular diffs Application server (data) openstreetmap.org/ API or Web server Regular tile gen- eration (Mapnik) tiles

Client Client (e.g OpenLayers) ´ M GIS 52 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Mapnik

A (standalone) tile generator for OpenStreetMap: Mapnik

OSM file OR Mapnik Tile (image)

PostGIS database

´ M GIS 53 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete OSM API v0.6

I RESTFul API to consult and edit OSM entities

I requests take the form of HTTP GET, PUT, POST, and DELETE messages

I requests return or expect the data for the entities in a XML format http://wiki.openstreetmap.org/wiki/OSM_Protocol_Version_0.6/DTD

I Example : GET /api/0.6/[node|way|relation]/#id

I returns the XML representation of the entity

´ M GIS 54 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete OSM API v0.6

http://api.openstreetmap.org/api/0.6/way/23000671

[...]

´ M GIS 55 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete OSM API v0.6

http://api.openstreetmap.org/api/0.6/node/344548301

´ M GIS 56 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Overpass API

http://wiki.openstreetmap.org/wiki/Overpass_API

I an optimized read-only API that serves up custom selected parts of the OSM map data

I a powerful query language with search criteria like e.g. location, type of objects, tag properties, proximity, or combinations of them.

´ M GIS 57 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Overpass API: Example

Overpass XML Overpass QL

relation ["name"="Ensimag"] ; (._; >; ); out

OR

Output in OSM XML or JSON

´ M GIS 58 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Overpass API: Example http://overpass-api.de/api/interpreter?data=relation["name"="Ensimag"](45. 13,5.67,45.22,5.78);(._;>;);out;

The data included in this document is from www.openstreetmap.org. The data is made available under ODbL. [...] [...] [...] [...]

´ M GIS 59 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Overpass API Applications

http://wiki.openstreetmap.org/wiki/Overpass_API/Applications

Overpass Turbo: A web based graphical user interface for Overpass API

http://overpass-turbo.eu/

´ M GIS 60 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete http://www.overpass-api.de/api/sketch-line?network=TAG&ref=B&correspondences=100& width=1600&force-rows=1 ⇓

Overpass API Applications

Public transport line generator http://www.overpass-api.de/public_transport.html

´ M GIS 61 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete ⇓

Overpass API Applications

Public transport line generator http://www.overpass-api.de/public_transport.html

http://www.overpass-api.de/api/sketch-line?network=TAG&ref=B&correspondences=100& width=1600&force-rows=1

´ M GIS 61 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Overpass API Applications

Public transport line generator http://www.overpass-api.de/public_transport.html

http://www.overpass-api.de/api/sketch-line?network=TAG&ref=B&correspondences=100& width=1600&force-rows=1 ⇓

´ M GIS 61 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Nominatim

I Nominatim: tool to search OSM data by name and address.

I Accessible through the HTTP protocol with GET parameters → can be queried with a standard web browser or with command-line tools.

I Three output formats:

I Standard HTML (a web page with embedded openlayers map)

I XML

I JSON

Example:

[sylvain@msnordlys]~ $ curl "http://nominatim.openstreetmap.org/search.php?q=rue+ de+la+Passerelle%2C+Saint-Martin-d%27H%C3%A8res&polygon=1&format=xml"

´ M GIS 62 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Nominatim: result of the query

´ M GIS 63 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete End user applications

I MapOSMatic

I WheelMap

I OpenRouteService: http://openrouteservice.org/

I “Thematic” maps:

I http://openpistemap.org/ – map of skiing/snowboarding pistes

I http://opencyclemap.org/ – map of cycling routes

I http://openseamap.org/ – map of sea navigation elements

´ M GIS 64 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Using OpenStreetMap data

Exploiting the data Retrieving data (basic principles) OSM API Other querying tools End user applications

Tutorial: create your own database Basic principles A step by step example

´ M GIS 65 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Data formats

Where to download ?

I http://download.geofabrik.de/osm/ : OSM files (XML format) by geographical area, city, country. . .

I http://download.cloudmade.com/ : Garmin Map files, Shape files, TomTom POI, Adobe Illustrator, ..., by geographical area, city, ...

I http://planet.osm.org/ : OSM files + changesets.

What to download ?

I OSM files: planet.osm for the entire planet (currently over 27GB compressed, over 300GB uncompressed), or regional extracts.

I Diffs: changesets for regular database updates (example: weekly changesets in http://planet.osm.org/ ).

´ M GIS 66 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Create your own database

PostGIS OSM file database

Several possible tools and schemas. . .

I Osm2pgsql: Lossy translation (only converts some entities). Reconstructs polygons from relations (e.g for administrative boundaries)

I Osmosis: OSM data general purpose processing tool

I converts OSM data to Postgres/PostGIS DB (sticks to OSM datamodel)

I generates planet dumps from a DB

I applies or generates changesets

I extracts data. . .

´ M GIS 67 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete A small Example

The problem I want to develop a little application that displays a map showing the French departments and regions, and the French road network.

My initial configuration: a PC running Ubuntu 12.10 GNU/Linux OS

1st step. Install Postgresql:

[sylvain@msnordlys]~ $ sudo apt-get install postgresql-9.1

´ M GIS 68 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete PostGIS installation

2nd step. Install PostGIS:

[sylvain@msnordlys]~ $ sudo apt-get install postgis postgresql-9.1-postgis

(A piece of cake... We have some packages for that in the distribution)

´ M GIS 69 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Database creation

3rd step. Login as postgres user and create a new user and a new database (and set this new user to be the database’s owner):

[sylvain@msnordlys]~ $ sudo -s Password: [root@msnordlys]~ # su postgres [postgresql@msnordlys]~ $ psql postgres=# CREATE USER [username]; CREATE ROLE postgres=# ALTER USER [username] WITH ENCRYPTED PASSWORD ’[password]’; ALTER ROLE postgres=# CREATE DATABASE [dbname]; CREATE DATABASE postgres=# ALTER DATABASE [dbname] OWNER TO [username]; ALTER DATABASE

´ M GIS 70 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete PostGIS Enabling

4th step. Enable PostGIS spatial functions into the database:

[postgresql@msnordlys]~ $ cd /usr/share/postgresql/9.1/contrib/postgis-1.5 [postgresql@msnordlys]/.../postgis-1.5 $ createlang plpgsql mydb [...] [postgresql@msnordlys]/.../postgis-1.5 $ psql -d [dbname] -f postgis.sql [...] [postgresql@msnordlys]/.../postgis-1.5 $ psql -d [dbname] -f spatial_ref_sys.sql [...] [postgresql@msnordlys]/.../postgis-1.5 $ cd .. [postgresql@msnordlys]/.../postgis-1.5 $ psql -d [dbname] -f postgis_comments.sql [...]

´ M GIS 71 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Data conversion

5th step. Run osm2pgsql to load Data into the database:

[sylvain@msnordlys]~ $ osm2pgsql -U [username] -d [dbname] -r pbf --cache=4000 \ > -W france.osm.pbf Password: Using projection SRS 900913 (Spherical Mercator) Setting up table: planet_osm_point NOTICE: table "planet_osm_point_tmp" does not exist, skipping Setting up table: planet_osm_line [...] Completed planet_osm_polygon

Osm2pgsql took 15962s overall

´ M GIS 72 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete A test request

We can now check whether the data has been correctly loaded...

[sylvain@msnordlys]~ $ psql -U [username] -d [dbname]

Password for user [username]: psql (9.1.8) Type "help" for help.

[dbname]=# SELECT name, place, ST_XMin(way), ST_YMin(way) [dbname]-# FROM planet_osm_point WHERE name=’Grenoble’ AND place=’city’; name | place | st_xmin | st_ymin ------+------+------+------Grenoble | city | 638583.190179611 | 5650917.09875511 (1 row)

´ M GIS 73 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Boundaries and roads

The SQL requests inside the program...

Retrieve administrative boundaries

SELECT name, ST_SimplifyPreserveTopology(way,5000), admin_level FROM planet_osm_polygon WHERE boundary=’administrative’AND admin_level <=’6’;

Retrieve the road network

SELECT ST_SimplifyPreserveTopology(way,5000), highway FROM planet_osm_line WHERE highwayIN(’motorway’,’trunk’,’primary’,’secondary’);

´ M GIS 74 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete The result

´ M GIS 75 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Fourth part

4 Using Volunteered Geographic Information

Authoritative data vs participative data

Authoritative data, conventional data

About metadata

Can we trust OpenStreetMap?

About data quality

Some OSM data quality tools

Case 1: Data quality for Location Based Services

Case 2: Evaluating OSM data quality on the Department of Sarthe Using Volunteered Geographic Information

Authoritative data vs participative data Authoritative data, conventional data About metadata

Can we trust OpenStreetMap? About data quality Some OSM data quality tools Case 1: Data quality for Location Based Services Case 2: Evaluating OSM data quality on the Department of Sarthe

´ M GIS 77 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete “VGI is not a very important trend in GIS nowadays, so we do not consider this approach in our GIS department”

(Head of GIS department of a big IT group — cited from memory)

I Are participative and authoritative data production so antagonistic?

I Can we trust participative data?

VGI vs authoritative

“My project [put here the name of any serious research or industrial project] cannot use VGI data because we cannot control, trust, verify [put any other reason here] it...”

(Anonymous quotation)

´ M GIS 78 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I Are participative and authoritative data production so antagonistic?

I Can we trust participative data?

VGI vs authoritative

“My project [put here the name of any serious research or industrial project] cannot use VGI data because we cannot control, trust, verify [put any other reason here] it...”

(Anonymous quotation)

“VGI is not a very important trend in GIS nowadays, so we do not consider this approach in our GIS department”

(Head of GIS department of a big IT group — cited from memory)

´ M GIS 78 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete VGI vs authoritative

“My project [put here the name of any serious research or industrial project] cannot use VGI data because we cannot control, trust, verify [put any other reason here] it...”

(Anonymous quotation)

“VGI is not a very important trend in GIS nowadays, so we do not consider this approach in our GIS department”

(Head of GIS department of a big IT group — cited from memory)

I Are participative and authoritative data production so antagonistic?

I Can we trust participative data?

´ M GIS 78 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete According to Van der Molen and Wubbe [2007] (cited by Coleman [2013]):

I High-quality database

I Explicit guarantees about quality assurance

I Contains essential data about persons, institutions, issues, entities...

I Designated by law as the sole officially recognized register to be used by government agencies

Coleman, D. J. (2013). Potential contributions and challenges of VGI for conventional topgraphic base-mapping programs. In Crowdsourcing Geographic Knowledge [...], chapter 14. Springer.

Van Der Molen, P. and Wubbe, M. (2007). E-government and e-land administration-as an example: The netherlands. In 6th FIG Regional Conference, San Jose, Costa Rica, pages 12–15.

Authoritative data

Authoritative data ≈ produced by professional mapping organizations

´ M GIS 79 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Authoritative data

Authoritative data ≈ produced by professional mapping organizations According to Van der Molen and Wubbe [2007] (cited by Coleman [2013]):

I High-quality database

I Explicit guarantees about quality assurance

I Contains essential data about persons, institutions, issues, entities...

I Designated by law as the sole officially recognized register to be used by government agencies

Coleman, D. J. (2013). Potential contributions and challenges of VGI for conventional topgraphic base-mapping programs. In Crowdsourcing Geographic Knowledge [...], chapter 14. Springer.

Van Der Molen, P. and Wubbe, M. (2007). E-government and e-land administration-as an example: The netherlands. In 6th FIG Regional Conference, San Jose, Costa Rica, pages 12–15.

´ M GIS 79 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Conventional data

The term authoritative is a bit restrictive Coote and Rackham [2008] prefer the term conventional data:

I Collected for specific purpose and requirements

I Usually not free (charged)

I Use limited to some organizations or individuals

I Copyrighted data

I Managed by organizations established for the purpose

I Collected by professional staff, paid for this

I Based on standard and established methods and practices

I Quality assessment at different levels, guarantees provided to the user

Coote, A. and Rackham, L. (2008). Neogeographic data quality — is it an issue? In AGI Geocommunity conference, ConsultingWhere Ltd.

´ M GIS 80 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Example – IGN, BD Topo

http://professionnels.ign.fr/bdtopo

I Restricted access, specific license (free for a sample, charged for the rest)

I IGN keeps the ownership of data, and only gives utilization rights

I Collected by professional staff at IGN, standard production techniques

I Complete metadata following ISO-19115 standard

I Well-documented dataset, detailed quality assessment

´ M GIS 81 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I Metadata: ≈ data about the data (producer, owner, units, error bounds, geographical bounding box, production process, source of data...)

I Usually, metadata is separated from data (ISO 19115)

I Usually, metadata production is separated from data production But in the context of VGI...

I Some metadata elements make sense for the dataset as a whole: owner (?), description of the dataset...

I But some don’t really: bounding box, producer, production process...

About metadata

One important aspect in Geographical datasets is metadata.

´ M GIS 82 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete But in the context of VGI...

I Some metadata elements make sense for the dataset as a whole: owner (?), description of the dataset...

I But some don’t really: bounding box, producer, production process...

About metadata

One important aspect in Geographical datasets is metadata.

I Metadata: ≈ data about the data (producer, owner, units, error bounds, geographical bounding box, production process, source of data...)

I Usually, metadata is separated from data (ISO 19115)

I Usually, metadata production is separated from data production

´ M GIS 82 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete I Some metadata elements make sense for the dataset as a whole: owner (?), description of the dataset...

I But some don’t really: bounding box, producer, production process...

About metadata

One important aspect in Geographical datasets is metadata.

I Metadata: ≈ data about the data (producer, owner, units, error bounds, geographical bounding box, production process, source of data...)

I Usually, metadata is separated from data (ISO 19115)

I Usually, metadata production is separated from data production But in the context of VGI...

´ M GIS 82 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete About metadata

One important aspect in Geographical datasets is metadata.

I Metadata: ≈ data about the data (producer, owner, units, error bounds, geographical bounding box, production process, source of data...)

I Usually, metadata is separated from data (ISO 19115)

I Usually, metadata production is separated from data production But in the context of VGI...

I Some metadata elements make sense for the dataset as a whole: owner (?), description of the dataset...

I But some don’t really: bounding box, producer, production process...

´ M GIS 82 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Reminder: Each OSM entity has:

I a numeric identifier: OSM ID

I a geometry

I a set of generic attributes present for every element

I uid, user: user id and user name

I timestamp: time of the last modification

I version: edit version of the object (starts from 1)

I changeset: the changeset (group of edits made within a certain time by one user) in which the object was created or updated

I comment: each changeset has an associated comment describing it

I a set of tags (attributes): key-value pairs Metadata, “embedded” in the description of the entity itself

Metadata embedded

Let’s have a look at OpenStreetMap...

´ M GIS 83 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Metadata, “embedded” in the description of the entity itself

Metadata embedded

Let’s have a look at OpenStreetMap...

Reminder: Each OSM entity has:

I a numeric identifier: OSM ID

I a geometry

I a set of generic attributes present for every element

I uid, user: user id and user name

I timestamp: time of the last modification

I version: edit version of the object (starts from 1)

I changeset: the changeset (group of edits made within a certain time by one user) in which the object was created or updated

I comment: each changeset has an associated comment describing it

I a set of tags (attributes): key-value pairs

´ M GIS 83 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Metadata embedded

Let’s have a look at OpenStreetMap...

Reminder: Each OSM entity has:

I a numeric identifier: OSM ID

I a geometry

I a set of generic attributes present for every element

I uid, user: user id and user name

I timestamp: time of the last modification

I version: edit version of the object (starts from 1)

I changeset: the changeset (group of edits made within a certain time by one user) in which the object was created or updated

I comment: each changeset has an associated comment describing it

I a set of tags (attributes): key-value pairs Metadata, “embedded” in the description of the entity itself

´ M GIS 83 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete But tags can also contain metadata, e.g.:

source = cadastre-dgi-fr source : Direction G´en´erale des Imp^ots - Cadastre. Mise `ajour : 2009 tiger:source = tiger_import_dch_v0.6_20070829 survey:date = 2013-12-10

Actually, some metadata is common to the whole OSM dataset: license, conditions of use, and OSM Wiki ( http://wiki.openstreetmap.org/ )

Metadata embedded

Metadata embedded: user, timestamp, version, changeset, comment

´ M GIS 84 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Actually, some metadata is common to the whole OSM dataset: license, conditions of use, and OSM Wiki ( http://wiki.openstreetmap.org/ )

Metadata embedded

Metadata embedded: user, timestamp, version, changeset, comment

But tags can also contain metadata, e.g.:

source = cadastre-dgi-fr source : Direction G´en´erale des Imp^ots - Cadastre. Mise `ajour : 2009 tiger:source = tiger_import_dch_v0.6_20070829 survey:date = 2013-12-10

´ M GIS 84 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Metadata embedded

Metadata embedded: user, timestamp, version, changeset, comment

But tags can also contain metadata, e.g.:

source = cadastre-dgi-fr source : Direction G´en´erale des Imp^ots - Cadastre. Mise `ajour : 2009 tiger:source = tiger_import_dch_v0.6_20070829 survey:date = 2013-12-10

Actually, some metadata is common to the whole OSM dataset: license, conditions of use, and OSM Wiki ( http://wiki.openstreetmap.org/ )

´ M GIS 84 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Centralized vs distributed approaches to metadata

I Traditional approach to metadata: “centralized” standards (ISO-19115)

I Clear standards, easy to navigate

I Heavy to produce ; sometimes incomplete even for authoritative data

I Does not always completely make sense

I Does not always completely reflects the production process

I Distributed “user-centric” approach:

I Can be messy

I Lightweight and flexible approach

I Consistent with the production process

´ M GIS 85 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete VGI and metadata

Conclusion about VGI and metadata [Poore and Wolf, 2013]: the process of producing metadata has to be rethought for VGI, because the dataset is:

I dynamically (continuously) generated

I generated locally by hundreds, thousands, or millions of users

; Production of metadata has to be integrated to the data production process

Poore, B. S. and Wolf, E. B. (2013). Metadata squared: Enhancing its usability for volunteered geographic information and the geoweb. In Crowdsourcing Geographic Knowledge [...], chapter 4. Springer.

´ M GIS 86 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Using Volunteered Geographic Information

Authoritative data vs participative data Authoritative data, conventional data About metadata

Can we trust OpenStreetMap? About data quality Some OSM data quality tools Case 1: Data quality for Location Based Services Case 2: Evaluating OSM data quality on the Department of Sarthe

´ M GIS 87 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete What is data quality?

Data quality is a perception or an assessment of data’s fitness to serve its purpose in a given context and a subjective to various applications. It highly depends on the need of individuals on how to use datasets.

[Caprioli et al., 2003]

Two main objectives:

I Is the dataset suitable for my needs?

I How can the dataset be improved?

Caprioli, M., Scognamiglio, A., Strisciuglio, G., and Tarantino, E. (2003). Rules and standards for spatial data quality in gis environments. In Proc. 21st Int. Cartographic Conf., pages 10–16, Durban, South Africa.

´ M GIS 88 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Main dimensions

I Accuracy Does it reflect reality? [Olson, 2003]

I Completeness Is something missing? [Batini and Scannapieco, 2006],

I Timeliness Is the data up-to-date? [Pipino et al., 2002]

I Volatility How long stays the information valid? [Batini and Scannapieco, 2006]

Batini, C. and Scannapieco, M. (2006). Data quality: concepts, methodologies and techniques. Springer. Olson, J. E. (2003). Data quality: the accuracy dimension. Morgan Kaufmann. Pipino, L. L., Lee, Y. W., and Wang, R. Y. (2002). Data quality assessment. Communications of the ACM, 45(4):211–218.

´ M GIS 89 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Main dimensions

I Consistency Does it involve inconsistent data? [Batini and Scannapieco, 2006]

I Quality of information source Credibility? [Pipino et al., 2002]

I Validity Inside a range? [Olson, 2003]

I Understandability Clarity of information? [Pipino et al., 2002]

Batini, C. and Scannapieco, M. (2006). Data quality: concepts, methodologies and techniques. Springer. Olson, J. E. (2003). Data quality: the accuracy dimension. Morgan Kaufmann. Pipino, L. L., Lee, Y. W., and Wang, R. Y. (2002). Data quality assessment. Communications of the ACM, 45(4):211–218.

´ M GIS 90 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Geographical dimensions [ANZLIC, 2001]

I Lineage Can we assess where the data comes from?

I Positional accuracy Absolute Position

I Attributive accuracy Does it contain all attributes to reflect reality?

I Logical consistency Does it contain logical errors?

I Completeness Does it contain all information needed for the task at hand?

ANZLIC (2001). Anzlic metadata guidelines: Core metadata elements for geographic data in australia and new zealand. Technical report, ANZLIC.

´ M GIS 91 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Assessing geographical data Quality

I Ground truth

I Ground truth on sparse points

I Second dataset considered as ground truth

I Logical rules, mainly for checking logical consistency (open polygons, non-crossing roads, etc.) or completeness (missing mandatory information)

I Statistical approach using predefined metrics based e.g. on completeness of information or user activity

´ M GIS 92 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Bug reporting

http://www.openstreetmap.org/

I Anyone can add notes to any geographical place on the OSM website

I Notes can be visualized on the map or downloaded as RSS feeds using OSM API

http://www.openstreetmap.org/api/0.6/notes/feed?bbox=5.7393265,45.1798589,5. 7891083,45.206352

´ M GIS 93 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Error detection tools

I Keep Right ( http://keepright.at/ )

I Detects: non-closed areas, dead-ended one-ways, almost junctions, deprecated tags, missing tags, bridges/tunnels without layer (careful - not always an error), motorways without ref, places of worship without religion, POIs without name, ways without nodes, floating islands, un-tagged railway crossings, wrongly-used railway crossing tag, objects with FIXME tags, and relations without type.

I Osmose ( http://osmose.openstreetmap.fr/map/ )

I Similar to KeepRight. I Currently, it covers :

I France I some nearby countries in Europe: Belgium, Luxembourg, and Switzerland; I some nearby countries in Africa/Indian Ocean: Madagascar, Cameroon

´ M GIS 94 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Case 1: Data quality for Location Based Services

Context: Data Quality Analysis of the OpenStreetMap Project Regarding Location Based Services, Bachelor Thesis by Niklas Petersen

Approach: 2 I Grid Analysis with 1km cells

I Count the number of values in each cell

I Merge with population data (reference ≈ ground truth)

I Highlight differences

´ M GIS 95 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete House numbers

´ M GIS 96 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete House numbers

´ M GIS 97 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Buildings

´ M GIS 98 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Roads

´ M GIS 99 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Shops

´ M GIS 100 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Conclusion about the case study

I Urban > rural regions

I Strongest coverage:

I Buildings: Netherlands → can be considered as complete

I Housenumbers: Czech Republic → coverage in general weak

I Shops: Switzerland → big difference between countries

´ M GIS 101 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Case 2: Evaluating OSM data quality on the Department of Sarthe

Context: a study of OSM data quality carried out by Petit, Billon and Follin [2012] on the French department of Sarthe

OSM data analyzed from the point of view of:

I geometrical accuracy

I attributive accuracy

I completeness

Petit, O., Billon, P., and Follin, J.-M. (2012). Evaluation´ de la qualit´edes donn´ees OpenStreetMap sur la Sarthe et r´eflexion sur le processus de contribution. XYZ, (131):24–34.

´ M GIS 102 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Methodology

Reference: R´ef´erentiel `aGrande Echelle´ (RGE), IGN (authoritative data)

I Preprossessing: automatic matching of OSM and RGE road networks

I Completeness of road network: comparison of the total lengths OSM vs RGE, for each city

I Geometrical accuracy of road network: Hausdorff distance (roads) and average Euclidian distance (nodes)

I Attributive precision: comparison of lengths named (name = ...) roads between OSM and RGE + manual comparison to find the sources of mismatches (spelling errors, punctuation, incomplete names...)

I + Evaluation of the precision for two methods of contribution (GPS, and imagery-based digitizing)

´ M GIS 103 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Conclusions

I Heterogeneous data quality (urban > rural)

I Incompleteness is the main weakness (especially on rural areas) – to be compared with the homogeneity of RGE, whose mission is the complete cover of French territory

I Rather good geometrical accuracy

I A few attribute values but quite good accuracy

I Both digitizing methods are quite precise

´ M GIS 104 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Conclusion To VGI or not to VGI?

Conventional data VGI data

´ M GIS 106 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete To VGI or not to VGI?

Conventional data VGI data

Quality of data  Random quality 

´ M GIS 106 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete To VGI or not to VGI?

Conventional data VGI data

Quality of data  Random quality 

Guarantees about quality  No guarantee about quality 

´ M GIS 106 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete To VGI or not to VGI?

Conventional data VGI data

Quality of data  Random quality 

Guarantees about quality  No guarantee about quality 

Heavy production process  Lightweight production process 

´ M GIS 106 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete To VGI or not to VGI?

Conventional data VGI data

Quality of data  Random quality 

Guarantees about quality  No guarantee about quality 

Heavy production process  Lightweight production process 

Poor reactivity  Timeliness 

´ M GIS 106 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete To VGI or not to VGI?

Conventional data VGI data

Quality of data  Random quality 

Guarantees about quality  No guarantee about quality 

Heavy production process  Lightweight production process 

Poor reactivity  Timeliness 

Restrictions (usage)  Free, open 

´ M GIS 106 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete To VGI or not to VGI?

Conventional data VGI data

Quality of data  Random quality 

Guarantees about quality  No guarantee about quality 

Heavy production process  Lightweight production process 

Poor reactivity  Timeliness 

Restrictions (usage)  Free, open 

Conclusion: VGI is not the solution to everything, but certainly a solution

I If guarantees about data quality (with someone who is liable for that) is important, use conventional datasets

I If timeliness is more important (e.g. crisis management), consider using to VGI datasets

´ M GIS 106 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Is data quality compatible with agility?

Three possible approaches:

I Limit the number of authorized contributors (confidence network) ; Victoria Department of Sustainability and Environment, Australia

I Let the users annotate, then internally check the validity ; Google Map Maker, United States Geological Survey, 2001

I Mix participative and conventional data, flag participative data and let the end users choose whether they want to integrate them or not ; TomTom MapShare

VGI and conventional data

Actually, several traditional data producers consider integrating VGI in their data production process

´ M GIS 107 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete Three possible approaches:

I Limit the number of authorized contributors (confidence network) ; Victoria Department of Sustainability and Environment, Australia

I Let the users annotate, then internally check the validity ; Google Map Maker, United States Geological Survey, 2001

I Mix participative and conventional data, flag participative data and let the end users choose whether they want to integrate them or not ; TomTom MapShare

VGI and conventional data

Actually, several traditional data producers consider integrating VGI in their data production process

Is data quality compatible with agility?

´ M GIS 107 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete VGI and conventional data

Actually, several traditional data producers consider integrating VGI in their data production process

Is data quality compatible with agility?

Three possible approaches:

I Limit the number of authorized contributors (confidence network) ; Victoria Department of Sustainability and Environment, Australia

I Let the users annotate, then internally check the validity ; Google Map Maker, United States Geological Survey, 2001

I Mix participative and conventional data, flag participative data and let the end users choose whether they want to integrate them or not ; TomTom MapShare

´ M GIS 107 / 107 GdR MAGIS – Ecole de G´eomatique 29 septembre au 3 octobre 2014 – S`ete