Maa-123.2441 GIS Software Development 4 op Data and databases

10/27/2013 Jari Reini GIS – in 4 categories

•Categories based on competences – Data collection and management – Spatial data analysis – Software development – GIS Solutions

• Today concentrating on data collection, management and analysis (and partly SW development)

Jari Reini 10/27/2013 Architectural choices in GIS (simplified)

Desktop Client-Server Internet

Application Server Database

Jari Reini Database 10/27/2013 Principles

•Re-use, Keep it simple

•Use COTS as much as possible –Less in-house development –Less maintenance –Already includes a lot of required functionality

•For example –Storage: Oracle –Editing: ESRI ArcGIS –Quality checking: Radius, JavaML, JTS –Product Creation: FME and own tools

10/27/2013 Architecture and flow

Data Sourcing Editing Conversion Products

Public

Comm upload Delivery unity Database Server

Other Rule engine

Quality Pre-filter checks Specs

Jari Reini 10/27/2013 Content types •Maps (2D)

•Voice-Enabled Maps

•Address Points

•Points of Interest

•City Guides

•Landmarks

•3D maps

Jari Reini 10/27/2013 Content gategories

•Business content Guide –Office locations, factories, stores … -Voice Maps –Environmental data -Speed Cameras –Etc -Traffic •Find -Logistics –POIs Display –Address Points -2D/3D City Maps –Locality Index -3D Landmarks -3D Terrain Model

Jari Reini 10/27/2013 Tele Atlas:Mobile Mapping Vans

•Faster data collection •Superior accuracy •Advanced image collection for richer, more accurate content –Renders texture to buildings –Provides back up system for reported issues

10/27/2013 Databases

•PostgreSQL/PostGIS

•Oracle Spatial

•SQL Server / ArcSDE

•Etc

•Note: in exercises we're using only files (due to some IT issues), think OrbisGIS as a database

Jari Reini 10/27/2013 Concepts

•relational database management system (RDBMS) •a system for managing data stored in relations –relation is essentially a term for table –Table has a collection of rows –Row has a set of columns –Tables are grouped into databases •A more modern development is the object-oriented database

Jari Reini 10/27/2013 Example •CREATE TABLE cities ( –name varchar(80), –Int temperature –location point –);

•INSERT INTO cities VALUES ( –‘Helsinki', –-10, –'(60.16, 24.94)‘ –); •SELECT * from cities;

•UPDATE cities set temperature=3 where name=‘Helsinki’;

Jari Reini 10/27/2013 Spatial SQL

• the need for spatial query formalism has been clearly identified by and answered in – Spatial SQL (Max J. Egenhofer, 1994), ● geoPOM (Nittel and al., 1997), ● GeoSQL (Wang and al., 2000)... • SQL spatial has been standardized : – SQLMM : ISO/IEC 13249l – Specification for SQL, OGC • efficient implementations that spatially enabled RDBMS (PostgreSQL/PostGIS, Oracle Spatial, SQLite-spatial...).

Jari Reini 10/27/2013 OrbisGIS SQL

• Portable spatial processor. No RDMS needed • Deals with several formats • Generic datasource model • Multisources language : – able to deal with geometries, raster, TIN and Remote data (WFS, WCS). – Match SQL and OGC standards – Great extensibility and customization ability.

Jari Reini 10/27/2013 Put screenshots from spatiallanguage pdf

Jari Reini 10/27/2013 Jari Reini 10/27/2013 Geometry functions in databases

• Relationship – True or false • Processing – Gives you a new geometry • Measurement – Calculates something

Jari Reini 10/27/2013 Geometry relationship functions

•Distance(geometry, geometry) •Equals(geometry, geometry) Covers(geometry A, geometry B) •Disjoint(geometry, geometry) CoveredBy(geometry A, geometry B) •Intersects(geometry, geometry) Intersects(geometry, geometry) •Touches(geometry, geometry) Relate(geometry, geometry) •Crosses(geometry, geometry) •Within(geometry A, geometry B) •Overlaps(geometry, geometry) •Contains(geometry A, geometry B)

Jari Reini 10/27/2013 Geometry processing functions •Centroid(geometry) •Area(geometry) •PointOnSurface(geometry) •Boundary(geometry) •Buffer(geometry, double, [integer]) •ConvexHull(geometry) •Intersection(geometry, geometry) •SymDifference(B,A). •Union(geometry, geometry) •Union(geometry set)

10/27/2013 Measurement functions •ST_Area(geometry) •ST_distance_sphere(point, point) •ST_distance_spheroid(point, point, spheroid) •ST_length2d(geometry) •ST_length3d(geometry) •ST_length_spheroid(geometry,spheroid) •ST_length3d_spheroid(geometry,spheroid) •ST_distance(geometry, geometry) •ST_max_distance(linestring,linestring) •ST_perimeter(geometry) •ST_perimeter2d(geometry) •ST_perimeter3d(geometry) •ST_azimuth(geometry, geometry)

10/27/2013 Server programming

•Extending SQL

•Triggers

•PL/SQL

10/27/2013 GIS Objects

•"Simple Features" defined by the OpenGIS Consortium (OGC) –OGC "Simple Features for SQL" specification.

•Expression of spatial objects –Well -Known Text (WKT) –Well-Known Binary (WKB)

•include a spatial referencing system identifier (SRID)

10/27/2013 Examples: WKT

•POINT(0 0) •LINESTRING(0 0,1 1,1 2) •POLYGON((0 0,4 0,4 4,0 4,0 0),(1 1, 2 1, 2 2, 1 2,1 1)) •MULTIPOINT(0 0,1 2) •MULTILINESTRING((0 0,1 1,1 2),(2 3,3 2,5 4)) •MULTIPOLYGON(((0 0,4 0,4 4,0 4,0 0),(1 1,2 1,2 2,1 2,1 1)), (( -1 -1, -1 -2, - 2 -2,-2 -1,-1 -1))) •GEOMETRYCOLLECTION(POINT(2 3),LINESTRING((2 3,3 4)))

•INSERT INTO geotable ( the_geom, the_name ) • VALUES ( GeomFromText('POINT(63.34 23.39)', 312), 'A Place');

10/27/2013 SQL-MM Part 3

•SQL Multimedia Applications Spatial specification

•extends the simple features for SQL spec by defining a number of circularly interpolated curves

•SQL -MM used by many databases (e.g PostGIS, Oracle Spatial etc.)

10/27/2013 SQL-MM3 ST_Area ST_AsBinary ST_ExteriorRing ST_MPointFromText ST_SRID ST_GeometryN ST_MPointFromWKB ST_StartPoint ST_AsText ST_GeometryType ST_MPolyFromText ST_SymDifferenc ST_Boundary ST_GeomFromText ST_MPolyFromWKB e ST_Buffer ST_GeomFromWKB ST_NumGeometries ST_Touches ST_Transform ST_Centroid ST_InteriorRingN ST_NumInteriorRing ST_Intersection ST_NumPoints ST_Union ST_Contains ST_Intersects ST_OrderingEquals ST_Within ST_ConvexHull ST_IsClosed ST_Overlaps ST_WKBToSQL ST_CoordDim ST_IsEmpty ST_Perimeter ST_WKTToSQL ST_Crosses ST_IsRing ST_Point ST_X ST_Y ST_Difference ST_IsSimple ST_PointFromText ST_IsValid ST_PointFromWKB ST_Dimension ST_Length ST_PointN ST_Disjoint ST_LineFromText ST_PointOnSurface ST_Distance ST_LineFromWKB ST_PolyFromText ST_EndPoint ST_MLineFromText ST_PolyFromWKB ST_MLineFromWKB ST_Polygon ST_Envelope ST_Relate ST_Equals

10/27/2013 Loading/retrieving GIS data

•Loading –Using SQL –Using the loader

•Retrieving –Using SQL –Using dumper –Using e.g ODBC/JDBC

10/27/2013 Data Cities and countries

Shape files

10/27/2013 Databases: PostGIS

10/27/2013 PostGIS data upload and use

Create user



10/27/2013 PostGIS data upload and use

Convert and upload shape files using shp2pgsql  C:\Tools\PostgreSQL\8.3\bin>shp2pgsql.exe -c c:\temp\data-v1_1\countries.shp tkk.countries > countries.sql

10/27/2013 PostGIS data upload and use

C:\Tools\PostgreSQL\8.3\bin>psql -U tkk -d -f countries.sql

10/27/2013 10/27/2013 10/27/2013 PostGIS usage

Using SQL INSERT INTO gtest (ID, NAME, GEOM) VALUES ( 1, 'First Geometry', GeomFromText('LINESTRING(2 3,4 5,6 5,7 8)', -1));

SELECT * FROM geotable WHERE ST_DWithin(geocolumn, 'POINT(1000 1000)', 100.0);

10/27/2013 PostGIS from Java

•Use JDBC •Class.forName("org.postgresql.Driver"); String url = "jdbc:postgresql://localhost:5432/database";

conn = DriverManager.getConnection(url, "postgres", ""); Statement s = conn.createStatement(); ResultSet = s.executeQuery("select AsText(geom) as geom,id from geomtable")

10/27/2013 Oracle Spatial

Spatial operators

Network Management

Geocoder

Geo Raster

Topology

Spatial Data Mining

3D

10/27/2013 Geometric Types

Polygon Point Line string Polygon with hole

Self --crossing Self --crossing line strings polygons

Valid Not valid

10/27/2013 Spatial Data Model

Spatial layer

Geometries

Elements

Point Line Polygon

10/27/2013 Elements

•Basic building block of a geometry •Element types Element 6 –Point (1) Element 5 –Line (2) Element 4 –Polygon (3) Element 7 Element 2 •Constructed Element 1 using coordinates Element 3 •No limit on number of coordinates Hawaii , USA

10/27/2013 Geometry

Geometry 1 Geometry 2 California Texas •Represents a spatial feature •Consists of an ordered set of primitive elements Geometry 3 Geometry 4 •Uniquely identified Florida Hawaii with a geometry identifier

10/27/2013 Layers

•Consist of geometries that share a common set of attributes

States layer

10/27/2013 Spatial Indexing

•Provides an exclusive and exhaustive coverage of spatial objects •Indexes points, lines, and polygons •Fixed tiling or hybrid tiling

10/27/2013 Spatial Queries

•Issued through SQL •Contains a locational constraint, for example: –Find all insurance holders in the projected path of a hurricane –Find all ATMs in my area

10/27/2013 Spatial Joins

•A join of two layers based on the spatial component of the layers •Implemented using spatial index and spatial operators •Example: –Find all the wetlands within the national parks

10/27/2013 Java-based geometries

10/27/2013 Java Topology Suite (JTS)

•The JTS Topology Suite is an API of 2D spatial predicates and functions

•Based on Simple Features Specification for SQL (OGC)

10/27/2013 10/27/2013 Example

•Geometry g1 = new WKTReader().read("LINESTRING (0 0, 10 10, 20 20)");

•Geometry buffer = g1.buffer(100.0);

10/27/2013 Javasacript in geometries

10/27/2013 Geoscript

•http://www.geoscript.org

•GeoScript adds spatial capabilities to dynamic scripting languages – JavaScript – Python – Scala – Groovy

10/27/2013